7 replies to this topic

[mx3L]

Probably you already noticed, that there is some strange behaviour when you pause and then unpause your media files with comparison to pause/unpause on your recordings. By strange I mean that you can clearly see that it's not unpaused from the same frame.

This is caused by doing flushing seek to the same position for unpause - it means that we first clear pipeline, then driver's buffers and then we seek to the same position as was in paused, which will refill the pipeline again.

Problem is that as you can see it's really not the same position, and it's not done immediately.

 

Why is it done this way?

I'm not sure, maybe because in gstreamer-0.10 more source timed out after long time in paused state or maybe because one of sink's didn't preroll, which caused pipeline to stall?

 

How to improve it?

I suggest we can just unpause pipeline for selected sources will not timeout (filesrc, souphttpsrc), when in paused state.

 

What are the issues?

Sometimes one of our sinks doesn't preroll(when going to paused state sink wants to make sure that it can play something immediately, therefore it waits for first buffer to arrive on it's sinkpad), this means that whole pipeline is blocked since sink cannot commit its state and we are stuck.

 

Why is it happening?

This is still a question to answer, but we can prevent it from happening by turning off async state changes for our sinks. It means that sinks will not wait for preroll when going to paused and then commit their state, but will commit their state immediately, without worrying if they are prerolled or not.

 

Why is async state change turned on?

I'm not sure, this is first commit where it was added:

http://git.opendream...4093a0eb2247c30

 

When async state change is enabled, then when sink is going to paused state, it will return ASYNC state change and will wait until first buffer arives at its sinkpad, then it will commit state to PAUSED. This should be only important when we would synchronize buffers against clock, since sinks would only start to render when both were prerolled. I don't think we should care about this, we should fill driver buffers as fast as possible and let them do a synchronization.

 

What do you think?

 

Patches:

1. https://github.com/m...9b5492eed628b84

2. https://github.com/m...990da56931b70a6

3. https://github.com/m...74ed60a2a794e70

 

2. patch - probably this should be better set in servicemp3, so we don't force it in dvbmediasink

3. patch - is not neccessary but slightly improves seeking performance in vusolose(frames are not fastforward-ed after seeking for a few moments)

 

I did brief tests on vusolose on rtmp, http streams and local files, seeking pausing/unpausing, fastforward works flawlessly when all 3 patches were applied.

 

[athoik]

Hi @mx3L,

 

Regarding 1, it is possible to get a segmentation fault if gst_plugin_feature_get_name returns NULL.

 

-if (!strcmp(name, "filesrc") || !strcmp(name, "souphttpsrc"))
+if (name && (!strcmp(name, "filesrc") || !strcmp(name, "souphttpsrc")))

 

Patches seem ok!

 

Of course having some test scenarios would be nice (eg test files or streams).

[mx3L]

Hi,

Thank you for review of patches !

 

1. You're right, patch updated - https://github.com/m...1deadfc5cb656b8

 

Currently I'm reproducing not preroll issue consistently on one http source, (with 2.patch there is no issue). It's possible to reproduce also on filesrc, but it's quite random(pause/unpause/pause/unpause... in quick succession).

I will try to find something where it's easy to reproduce and post it here..

[MastaG]

mx3L thanks for your patches.

Does the servicemp3.cpp patch depend on the patches for the dvbmediasink?

If yes, then maybe christophecvr can also apply it onto his multibox git.

[mx3L]

@MastaG

No it doesn't, but I'm not sure if it will have any effect since christophecvr mediasink already does this, since you cannot get playposition in paused state -> no valid position in servicemp3 -> no seeking, just unpause exactly as servicemp3 patch but for all sources.

[christophecvr]

@MastaG

No it doesn't, but I'm not sure if it will have any effect since christophecvr mediasink already does this, since you cannot get playposition in paused state -> no valid position in servicemp3 -> no seeking, just unpause exactly as servicemp3 patch but for all sources.

Once xml patches are pushed to enigma2 , I will check If I can eventually remove the play stop up on pause whitout triggering a not needed seek up on unpause.

 

The main problem was that a flush seek was done for audio only not for video and then it was out off sink.

Edited by christophecvr, 9 September 2015 - 12:10.

[christophecvr]

Some more details about the main cause off this issue.

 

First we need to run the audiosink and videosink asynchrone. This requires the flush seek and passing off segment to both sinks simultaneously.

 

The majority off movie containers do require to have the video and audio in sink (within certain limits). One off the bugs by gst-0.10 is that this limit was to small for mkv containers.

 

What I did in the multisink, was since by pause the video stopped direct but not the audio. Stop the audio self by setting self->playing = false. The result is that the audio stopped also where it was . No position was taken anymore.  By resume the video resumed always without flushing or whitout a new seek and resulting segment. Gstreamer position was equal to that from the dvb. But the audio had a position later then the video.

 

Result video resumed where it stopped. The audio tried to resume with a new segment from later and it was all out off sink. The container mkv(m4v) stopped the audio due to that.

 

Now in my sink I just stopped the audio at the same time like the video. No position is fetched anymore, resulting in a unknown seek position and no seek is done for audio.

Up on unpause the audio will resume togheter with the video and the dvb positions where equal to the gstreamer position. Result al is still in sink and plays fine.

 

Note containers like the wmv on a dm8000 at least for the old media needs a manual user intervention by setting a delay. They do not require the audio and video to be in sink.

On vuduo2 where wmv needs to be done via gstreamer it needs to be in sink. But gstreamer take cares about it as long we do not have that pause/unpause issue.

[mx3L]

So I tried to figure out why exactly one of the sinks didn't preroll, here are my findings:

 

sample:

https://dl.dropboxus...38760017/bb.avi

 

picture of pipeline:

https://dl.dropboxus...17/pipeline.png

 

debug output:

https://dl.dropboxus...bug_nocolor.log

unsuccessfull unpause - lines 58106-58146

 

gdb output when going successfully to paused state:

http://pastebin.com/vV45iYL5

 

6.thread - pulling from aqueue pushing to dvbaudiosink

5.thread - pulling from vqueue pushing to dvbvideosink

4.thread - pulling from multiqueue ..pushing to aqueue

3.thread - pulling from multiqueue ..pushing to vqueue

2.thread - ..pushing from avidemux to multiqueue

 

(6,5) sinks have prerolled buffer, and are waiting for change state to playing, which will unlock both chains

(3,4) vqueue and aqueue are filled and threads are waiting for space in these queues to push next buffers to them

2 multiqueue is filled(probably video part), 2.thread is waiting for space in multiqueue to push buffers from avidemux

 

gdb output when going un-successfully to paused state:

http://pastebin.com/PXuq78Q4

 

6. audiosink is not prerolled and is waiting for buffer from empty aqueue

5. videosink is prerolled and is blocking video chain

4. aqueue is empty and is waiting for buffer from multiqueue which is also empty(audio part)

3. vqueue is filled as is multiqueue(videopart)

2. avidemux wants to push video buffer to multiqueue(video part) which is filled(avidemux pushes audio/video buffers sequentially at least that's how it looks like from log)

 

So avidemux wants to push video buffer to multiqueue(video part) which is filled and cannot be freed, since video chain is blocked by videosink -> we cannot refill empty multiqueue(audio part), so audiosink will wait indefinitely for preroll buffer

 

Why is it happening?

We have unbalanced flow of audio/video buffers, because of async setting and different size of video/audio buffers.

So video buffers will fill much faster video hw buffer then audio buffers will fill hw audio buffer (hw buffers have same size)?

 

For example on unsuccessfull unpause, we just pushed to dvbaudiosink buffer with 1:28 and videosink has time only 1:08:

0:04:48.608164902 ^[[334m 2283   0x5118c0 DEBUG               basesink gstbasesink.c:2539:gst_base_sink_do_sync:<dvbaudiosink0> possibly waiting for clock to reach 0:01:28.440000000, adjusted 0:01:28.440000000
0:04:48.608731680 ^[[334m 2283   0x5118c0 DEBUG               basesink gstbasesink.c:2167:gst_base_sink_wait_clock:<dvbaudiosink0> sync disabled
0:04:48.609026346 ^[[334m 2283   0x5118c0 DEBUG               basesink gstbasesink.c:2546:gst_base_sink_do_sync:<dvbaudiosink0> clock returned 4, jitter  0:00:00.000000000
0:04:48.609512383 ^[[334m 2283   0x5118c0 DEBUG               basesink gstbasesink.c:3501:gst_base_sink_chain_unlocked:<dvbaudiosink0> rendering object 0x76bc5a88
0:04:48.610206865 ^[[334m 2283   0x5118c0 DEBUG               basesink gstbasesink.c:946:gst_base_sink_set_last_buffer_unlocked:<dvbaudiosink0> setting last buffer to 0x76bc5a88
0:04:48.611024420 ^[[334m 2283   0x5118c0 DEBUG               basesink gstbasesink.c:3547:gst_base_sink_chain_unlocked:<dvbaudiosink0> object unref after render 0x76bc5a88
0:04:48.775547457 ^[[334m 2283 0x76004520 DEBUG               basesink gstbasesink.c:5113:gst_base_sink_change_state:<dvbaudiosink0> PLAYING to PAUSED
0:04:48.775965605 ^[[334m 2283 0x76004520 DEBUG           dvbaudiosink gstdvbaudiosink.c:346:gst_dvbaudiosink_unlock:<dvbaudiosink0> unlock
0:04:48.776222013 ^[[334m 2283 0x76004520 DEBUG               basesink gstbasesink.c:5122:gst_base_sink_change_state:<dvbaudiosink0> got preroll lock
0:04:48.777494568 ^[[334m 2283 0x76004520 DEBUG           dvbaudiosink gstdvbaudiosink.c:354:gst_dvbaudiosink_unlock_stop:<dvbaudiosink0> unlock_stop
0:04:48.777794309 ^[[334m 2283 0x76004520 DEBUG               basesink gstbasesink.c:3313:gst_base_sink_needs_preroll:<dvbaudiosink0> have_preroll: 0, EOS: 0 => needs preroll: 1
0:04:48.778528531 ^[[334m 2283 0x76004520 DEBUG               basesink gstbasesink.c:5144:gst_base_sink_change_state:<dvbaudiosink0> PLAYING to PAUSED, we are not prerolled
0:04:48.778817161 ^[[334m 2283 0x76004520 DEBUG               basesink gstbasesink.c:5150:gst_base_sink_change_state:<dvbaudiosink0> doing async state change
0:04:48.779243976 ^[[334m 2283 0x76004520 DEBUG               basesink gstbasesink.c:5158:gst_base_sink_change_state:<dvbaudiosink0> rendered: 5, dropped: 0
0:04:48.779725791 ^[[334m 2283 0x76004520 DEBUG           dvbaudiosink gstdvbaudiosink.c:1540:gst_dvbaudiosink_change_state:<dvbaudiosink0> GST_STATE_CHANGE_PLAYING_TO_PAUSED
0:04:48.832513642 ^[[334m 2283   0x42db50 DEBUG               basesink gstbasesink.c:3547:gst_base_sink_chain_unlocked:<dvbvideosink0> object unref after render 0x76b9d148
0:04:48.833314346 ^[[334m 2283   0x42db50 DEBUG               basesink gstbasesink.c:3401:gst_base_sink_chain_unlocked:<dvbvideosink0> got times start: 0:01:08.680000000, end: 0:01:08.720000000

Eventually video hw buffer is filled and will start to block render in video-sink, which will cause quick filling of multiqueue(video part), while audio hw buffer is still not filled, which means that aqueue and multiqueue(audio part) is most of the time drained.

Now it's just matter of coincidence if we pause at the moment when avidemux tries to push video buffer to multiqueue(video part) when it's filled(video hw buffer is filled and is blocking) and multiqueue(audio part) is empty.

 

This is how I understand it.

 

Solution?

1. We can as suggested remove waiting for preroll, so video chain will not block. Problem is that drivers(at least vusolose) don't like when A/V buffers are far apart(picture freezed, while audio is playing). This is problem on start of playback(confirmed) or after flushing seek which will clear hw buffers(not happened yet). So this could be improved by doing initial wait for preroll when going from NULL to PAUSED and then turn it off, with this solution I didn't encouter any problems yet.

 

2. We can wait for preroll as it is right now, and treat this situation as special case in servicemp3. When audiosink cannot get it's preroll buffer, we will just unpause videoSink which will unlock videochain, so audio buffers can flow to multiqueue(audio-part) and audiosink will preroll. Currently I'm testing this solution and it looks to be working..

 

3. Probably the best solution would be to always wait for preroll buffer when we have clear hw buffers - on initial startup and on flushing seek. And not to wait for preroll when we are doing pause/unpause, since we are already prerolled in hw buffers.

4. ?

 

How to reproduce

download sample, play sample with gst-player, after ~10 seconds of playback(to fill hw buffers) start quickly pushing  "space"    which will pause/unpause pipeline, until you cannot unpause anymore..

Edited by mx3L, 10 September 2015 - 23:11.