2520 replies to this topic

[athoik]

Nice mx3l!

Patch for riff-media uploaded also on GStreamer bug for review.


@OpenPLi, do we want to use (master) git version for GStreamer bitbales or stable releases (eg 1.4.x)?

[pieterg]

I think, if patches (by you and mx3l) are accepted, it makes sense to use the git version (until they've made it into a stable release, at least)

[athoik]

Patch with fourcc is not accepted from GStreamer:

Sebastian Dröge (slomo) [GStreamer developer] 2014-11-07 09:31:46 UTC

(From update of attachment 290138 [details])
As said before this is not the right approach considering all the discussion we
had so far.

Your hardware probably needs different settings depending on the MPEG-4 Part 2
*profile* but not based on this magic fourccs. And mpeg4videoparse should
already add the profile to the caps.

So does profile works?

[mx3L]

That was quick

"profile" field is only added in caps for mpeg4 in mp4 video format and not for mpeg4 in avi, at least that's my experience

Edited by mx3L, 7 November 2014 - 10:41.

[Erik Slagter]

Yeah, rejects tend to be quick

 

Let's try to get the issue completely clear and work to a solution from there.

 

- avi cannot store B frames (because they need to be decoded out-of-order which avi doesn't have provisioning for)

- divx (xvid???) creates an mpeg4-vc elementary stream that has B frames "packed" with other (P? I sounds unlikely) frames

- these "packed" frames are stored in avi like one "normal" frame

- the avi demuxer, on decoding, only produces I and P frames (where P frames may be "packed" frames including one or more (?) B frames)

- if you feed a "packed" frame to the decoder, it won't work (what happens actually, does it only decode the first frame or nothing at all?)

- so at some point between demuxing and decoding, the "packed" frame needs to "unpacked" into P and B frames, where also the PTS and DTS timestamps are regenerated

- does gstreamer have functionality for that anyway?

 

I guess the crux is to determine if this something that should be handled by the decoder or by the demuxer. I think it should be handled by the decoder (the way it is conceived, this way), but apparently our hardware decoder doesn't like it that way. Also I am not sure that at the point the elementary stream is fed to the decoder, there is still enough context for the decoder to determine if the "packed" frames are used, or that this information is stored in the (avi) container. If the only hint whether "packed" frames are used is in the (avi) fourcc, then I think we're on very thin ice. Isn't there any other way to determine this fact, from other metadata (either container or elementary stream)?

 

And in that case, I guess a conversion layer should be added between demuxer and decoder, which gstreamer can do, apparently?

Edited by Erik Slagter, 7 November 2014 - 11:23.

[athoik]

A new bug was opened because mpeg4videoparse is not adding profile information: https://bugzilla.gno...g.cgi?id=739773
 

<athoik> slomo: if  mpeg4videoparse  was working on avi files adding profile (and other informations) will it add also information about bitstream (packed / unpacked) ?
<athoik> slomo:  packed/unpacked avi bitstream http://itsjustonesandzeros.blogspot.com/2007/01/what-is-packed-bitstream.html
<slomo> athoik: mpeg4videoparse should just unpack automatically
<slomo> athoik: you will get exactly one frame per buffer
<slomo> athoik: does that help?
<athoik> slomo: we are doing the opposite! we are packing the unpack bitstream (and until now nobody knows why this is done)
<slomo> athoik: well, that's a hardware specific thing then that you'll have to do in your sink :)

Edited by athoik, 7 November 2014 - 13:49.

[athoik]

Currently we are not getting codec_data (bug 739773) when using mpeg4videoparser with avi file.

Until this get fixed we can use it to detect most xvid avi...
 

Subject: [PATCH] dvbvideosink: when mpeg4 doesnt have codec_data then handle
 it as xvid


diff --git a/gstdvbvideosink.c b/gstdvbvideosink.c
index d8d5635..f23cec7 100755
--- a/gstdvbvideosink.c
+++ b/gstdvbvideosink.c
@@ -1304,11 +1304,19 @@ static gboolean gst_dvbvideosink_set_caps(GstBaseSink *basesink, GstCaps *caps)
 				{
 					GST_INFO_OBJECT (self, "MPEG4 have codec data");
 					self->codec_data = gst_value_get_buffer(codec_data);
-					self->codec_type = CT_MPEG4_PART2;
 					gst_buffer_ref (self->codec_data);
+					streamtype = 4;
 				}
-				streamtype = 4;
-				GST_INFO_OBJECT (self, "MIMETYPE video/mpeg4 -> VIDEO_SET_STREAMTYPE, 4");
+				else
+				{
+					/* HACK When mpeg4 doesn't have codec_data then most probably it's xvid */
+					streamtype = 10;
+#ifdef PACK_UNPACKED_XVID_DIVX5_BITSTREAM
+					self->must_pack_bitstream = TRUE;
+#endif
+				}
+				self->codec_type = CT_MPEG4_PART2;
+				GST_INFO_OBJECT (self, "MIMETYPE video/mpeg4 -> VIDEO_SET_STREAMTYPE, %d", streamtype);
 			}
 			break;
 			default:
-- 

When we have codec_data (after bug 739773 is fixed) then we need to find the magic bytes..
 

dvbvideosink gstdvbvideosink.c:1308:gst_dvbvideosink_set_caps:<dvbvideosink9> set_caps video/mpeg, mpegversion=(int)4, systemstream=(boolean)false, profile=(string)advanced-simple, level=(string)5, codec_data=(buffer)000001b0f5000001b5090000010000000120088685770307d31404221463000001b244697658353033623133393370000001b25876694430303634, width=(int)640, height=(int)272, framerate=(fraction)24000/1001, pixel-aspect-ratio=(fraction)1/1

codec_data=(buffer)000001b0f5000001b5090000010000000120088685770307d31404221463000001b2
44 69 76 58
353033623133393370000001b2
58 76 69 44 
30303634

 xvid.png   19.09KB   5 downloads

Most probably we can already use codec_data to detect xvid files in mp4 format and use correct streamtype...

Attached Files

•  0001-dvbvideosink-when-mpeg4-doesn-t-have-codec_data-then.patch   1.18KB   6 downloads

[mx3L]

 @Erik

if you feed a "packed" frame to the decoder, it won't work (what happens actually, does it only decode the first frame or nothing at all?)

It doesn't work with streamtype 4, but works with streamtype 10

 

 

If the only hint whether "packed" frames are used is in the (avi) fourcc, then I think we're on very thin ice. Isn't there any other way to determine this fact, from other metadata (either container or elementary stream)?

AVI with fourcc XVID doesn't mean that packed frames are used, It only means that there is posibility of packed frames. So we set streamtype 10(handles packed frames) for decoder and pack unpacked bitstream on the fly, metadata about packed/unpacked are in bitstream.

if (sscanf((char*)data+pos, "DivX%d%c%d%cp", &tmp1, &c1, &tmp2, &c2) == 4 && (c1 == 'b' || c1 == 'B') && (c2 == 'p' || c2 == 'P'))
                        {
                                GST_INFO_OBJECT (self, "%s seen... already packed!", (char*)data+pos);
                                self->must_pack_bitstream = FALSE;
                                break;
                        }

At least that's how I think it is.

 

 

@athoik

 

I don't think this is a bug. In dvbvideosink we don't require "profile" or "codec_data" field for mpeg4, so decodebin immediately chooses dvbvideosink instead of going through mpeg4videoparse. If we require "profile" field in dvbvideosink, it will go through mpeg4videoparse and it will append additional caps (codec_data, profile_field)

diff --git a/gstdvbvideosink.c b/gstdvbvideosink.c
index d8d5635..fdd015d 100755
--- a/gstdvbvideosink.c
+++ b/gstdvbvideosink.c
@@ -178,10 +178,12 @@ GST_STATIC_PAD_TEMPLATE (
        GST_STATIC_CAPS (
        "video/mpeg, "
 #ifdef HAVE_MPEG4
-               "mpegversion = (int) { 1, 2, 4 }, "
-#else
-               "mpegversion = (int) { 1, 2 }, "
+               "mpegversion = (int) 4, "
+               "profile = (string) { simple, advanced-simple }, "
+               VIDEO_CAPS "; "
 #endif
+       "video/mpeg, "
+               "mpegversion = (int) { 1, 2 }, "
                VIDEO_CAPS "; "
 #ifdef HAVE_H264
        "video/x-h264, "

gst-launch-1.0 -v playbin uri="file:///media/net/DOMA/Filmy/Unaveni slnkom/Unaveni-sluncem-2-Odpor-Cz.avi"|grep vbin
/GstPlayBin:playbin0/GstPlaySink:playsink/GstBin:vbin/GstDVBVideoSink:dvbvideosink0: ts-offset = 0
/GstPlayBin:playbin0/GstPlaySink:playsink/GstBin:vbin.GstGhostPad:sink.GstProxyPad:proxypad10: caps = "video/mpeg\,\ mpegversion\=\(int\)4\,\ systemstream\=\(boolean\)false\,\ fourcc\=\(uint\)1145656920\,\ framerate\=\(fraction\)25/1\,\ width\=\(int\)720\,\ height\=\(int\)304\,\ parsed\=\(boolean\)true\,\ profile\=\(string\)advanced-simple\,\ level\=\(string\)5\,\ codec_data\=\(buffer\)000001b0f5000001b509000001000000012008868400670c5a1098518f\,\ pixel-aspect-ratio\=\(fraction\)1/1"

[Erik Slagter]

It doesn't work with streamtype 4, but works with streamtype 10
 
AVI with fourcc XVID doesn't mean that packed frames are used, It only means that there is posibility of packed frames. So we set streamtype 10(handles packed frames) for decoder and pack unpacked bitstream on the fly, metadata about packed/unpacked are in bitstream.

Streamtypes??? Yet another avi aborration?

So I am right in that the "fourcc" in itself doesn't signify "packed" format or not?

mpeg4-vc doesn't specify a "packed" format for it's elementary stream, so it will have to converted at some point before feeding it to the decoder, but I guess I've said that a few times before.

[Sjaaky]

 

It doesn't work with streamtype 4, but works with streamtype 10
 

Streamtypes??? Yet another avi aborration?

 

 

No, just a nummer to tell the driver and/or hardware what kind of video is fed to it. 

[athoik]

The facts from GStreamer 0.10:

When XviD packed then streamtype = 10
When XviD unpacked then streamtype = 10 and must_pack_bitstream = true

diff --git a/gstdvbvideosink.c b/gstdvbvideosink.c
index d8d5635..a7ff0f7 100755
--- a/gstdvbvideosink.c
+++ b/gstdvbvideosink.c
@@ -150,6 +150,49 @@ void bitstream_put(struct bitstream *bit, unsigned long val, int bits)
 }
 #endif
 
+gboolean is_xvid(size_t size, char *data, int *build)
+{
+	size_t pos = 0;
+	while (pos < size)
+	{
+		if (memcmp(&data[pos], "\x00\x00\x01\xb2", 4))
+		{
+			pos++;
+			continue;
+		}
+		pos += 4;
+		if (size - pos < 8) break;
+		if (sscanf(data+pos, "XviD%d", build) == 1)
+		{
+			return TRUE;
+		}
+	}
+	return FALSE;
+}
+
+gboolean is_xvid_packed(size_t size, char *data)
+{
+	int tmp1, tmp2;
+	unsigned char c1, c2;
+	size_t pos = 0;
+	while (pos < size)
+	{
+		if (memcmp(&data[pos], "\x00\x00\x01\xb2", 4))
+		{
+			pos++;
+			continue;
+		}
+		pos += 4;
+		if (size - pos < 13) break;
+		if (sscanf(data+pos, "DivX%d%c%d%cp", &tmp1, &c1, &tmp2, &c2) == 4 && (c1 == 'b' || c1 == 'B') && (c2 == 'p' || c2 == 'P'))
+		{
+			return TRUE;
+		}
+	}
+	return FALSE;
+}
+
+
 GST_DEBUG_CATEGORY_STATIC (dvbvideosink_debug);
 #define GST_CAT_DEFAULT dvbvideosink_debug
 
@@ -176,12 +219,14 @@ GST_STATIC_PAD_TEMPLATE (
 	GST_PAD_SINK,
 	GST_PAD_ALWAYS,
 	GST_STATIC_CAPS (
-	"video/mpeg, "
 #ifdef HAVE_MPEG4
-		"mpegversion = (int) { 1, 2, 4 }, "
-#else
-		"mpegversion = (int) { 1, 2 }, "
+	"video/mpeg, "
+		"mpegversion = (int) 4, "
+		"profile = (string) { simple, advanced-simple }, "
+		VIDEO_CAPS "; "
 #endif
+	"video/mpeg, "
+		"mpegversion = (int) { 1, 2 }, "
 		VIDEO_CAPS "; "
 #ifdef HAVE_H264
 	"video/x-h264, "
@@ -792,27 +837,10 @@ static GstFlowReturn gst_dvbvideosink_render(GstBaseSink *sink, GstBuffer *buffe
 		}
 	}
 
-	if (self->must_pack_bitstream)
+	if (self->must_pack_bitstream && is_xvid_packed(data_len, data))
 	{
-		int tmp1, tmp2;
-		unsigned char c1, c2;
-		unsigned int pos = 0;
-		while (pos < data_len)
-		{
-			if (memcmp(&data[pos], "\x00\x00\x01\xb2", 4))
-			{
-				pos++;
-				continue;
-			}
-			pos += 4;
-			if (data_len - pos < 13) break;
-			if (sscanf((char*)data+pos, "DivX%d%c%d%cp", &tmp1, &c1, &tmp2, &c2) == 4 && (c1 == 'b' || c1 == 'B') && (c2 == 'p' || c2 == 'P')) 
-			{
-				GST_INFO_OBJECT (self, "%s seen... already packed!", (char*)data+pos);
-				self->must_pack_bitstream = FALSE;
-				break;
-			}
-		}
+		GST_INFO_OBJECT (self, "Xvid: already packed, must_pack_bitstream = false");
+		self->must_pack_bitstream = FALSE;
 	}
 #endif
 
@@ -1299,16 +1327,47 @@ static gboolean gst_dvbvideosink_set_caps(GstBaseSink *basesink, GstCaps *caps)
 			break;
 			case 4:
 			{
+				streamtype = 4;
+				self->codec_type = CT_MPEG4_PART2;
 				const GValue *codec_data = gst_structure_get_value(structure, "codec_data");
 				if (codec_data)
 				{
 					GST_INFO_OBJECT (self, "MPEG4 have codec data");
 					self->codec_data = gst_value_get_buffer(codec_data);
-					self->codec_type = CT_MPEG4_PART2;
 					gst_buffer_ref (self->codec_data);
+#if GST_VERSION_MAJOR >= 1
+					GstMapInfo map;
+					gst_buffer_map(self->codec_data, &map, GST_MAP_READ);
+					gboolean parsed;
+					if(is_xvid(map.size, (char*)map.data, &mpegversion))
+					{
+						streamtype = 10;
+						GST_INFO_OBJECT (self, "XviD: build %04d detected, using streamtype %d", mpegversion, streamtype);
+#ifdef PACK_UNPACKED_XVID_DIVX5_BITSTREAM
+						if(is_xvid_packed(map.size, (char*)map.data))
+						{
+							GST_INFO_OBJECT (self, "XviD: packed bitstream detected, must_pack_bitstream = false");
+							self->must_pack_bitstream = FALSE;
+						}
+						else
+						{
+							GST_INFO_OBJECT (self, "XviD: unpacked bitstream detected, must_pack_bitstream = true");
+							self->must_pack_bitstream = TRUE;
+						}
+#endif
+					}
+					else if(gst_structure_get_boolean(structure, "parsed", &parsed) && parsed)
+					{
+						streamtype = 10;
+						GST_INFO_OBJECT (self, "XviD: parsed detected, using streamtype %d, must_pack_bitstream = true", streamtype);
+#ifdef PACK_UNPACKED_XVID_DIVX5_BITSTREAM
+						self->must_pack_bitstream = TRUE;
+#endif
+					}
+					gst_buffer_unmap(self->codec_data, &map);
+#endif
 				}
-				streamtype = 4;
-				GST_INFO_OBJECT (self, "MIMETYPE video/mpeg4 -> VIDEO_SET_STREAMTYPE, 4");
+				GST_INFO_OBJECT (self, "MIMETYPE video/mpeg4 -> VIDEO_SET_STREAMTYPE, %d", streamtype);
 			}
 			break;
 			default:

Now that we get parsed (because we request profile in caps, and that is correct ofc) I cannot understand the behaviour.

When avi is parsed, I think it is removing packed bitstream...

Please have a look, I run out of ideas...

Attached Files

•  dvbmediasink_xvid_mpeg4.patch   4.22KB   2 downloads

[athoik]

Here is something new!

Unpacked xvid avi play just fine with streamtype = 4 and ofc must_pack_bitstream = false!

So do we still need the hack?

PS. It takes only few seconds to unpack an xvid avi (transfer might take longer ), use unpackmp4 from here http://www.xmixdrix..../unpackmp4.html

[betacentauri]

It would be nice if we can handle automatically both packed and unpacked xvid. How should we inform the user that they should unpack their videos? So we "only" need to know whether we have a packed/unpacked xvid. I already try to find a solution by looking into mediainfo source code.

[athoik]

Mediainfo will look for userdata, look above at is_xvid_packed and is_xvid. I don't think it really scans the frames to detect if the file is packed or unpacked.

I suggest to look at the unpackmp4 source code (http://www.xmixdrix....packmp4-src.zip).

At least please verify that unpacked avi will work fine with gstreamer using streamtype = 4 (without any patches actually!, dts and pts patch is not required and with that applied it doesn't work)

Edited by athoik, 9 November 2014 - 15:08.

[betacentauri]

Mediainfo scans the file and checks whether there are 2 frames in one block.

http://sourceforge.n...Mpeg4v.cpp#l559

So I think it's quite similar to unpackmp4 source.

 

Do we always get all data for one frame in gstdvbvideosink? Then we can scan the data whether there are 2 vops defined.

 

But I see a problem. First frame is a I frame. We have to decide immediately whether we use streamtype 4 or 10. I guess it makes problems when we write I frame with streamtype 4 and then switch to 10 because data is packed.

Or we need to buffer 2-3 frames in the sink

Edited by betacentauri, 9 November 2014 - 15:19.

[athoik]

I guess we need to find a way in gst_mpeg4vparse_parse_frame :
http://cgit.freedesk...deoparse.c#n627

[athoik]

New patch, this one now should detect packed avi and set streamtype = 10 all other should work with streamtype = 4.

In caps we dont have the details required to detect packed avi, but in gst_dvbvideosink_render we do...

So if we haven't written the pts yet (first time that gst_dvbvideosink_render is called) and we have xvid with packed data, switch to streamtype = 10!

	if (!self->pts_written) hexdump(data, data_len);
	int build;
	if (self->codec_type == CT_MPEG4_PART2 && !self->pts_written && is_xvid(data_len, data, &build) && is_xvid_packed(data_len, data))
	{
		if(ioctl(self->fd, VIDEO_SET_STREAMTYPE, 10) < 0)
			GST_INFO_OBJECT (self, "XviD: packed detected, setting streamtype to 10 failed!!!");
		else
			GST_INFO_OBJECT (self, "XviD: packed detected, setting streamtype to 10");
	}

but again packed xvid is not working

Attached Files

•  dvbmediasink_xvid_mpeg4_try2.patch   7.52KB   9 downloads
•  packed.log   15.26KB   5 downloads
•  unpacked.log   14.85KB   2 downloads

[betacentauri]

I guess checking meta informations is not enough. Encoder can write whatever they want... And some people patch such informations.

[urkagan]

Problem with OpenPli OE 2.0 media player. Does not go back to the video list, and does not switch to the next movie.

only .ts none problems. Help me.

[athoik]

Could you please check if the following patch fixes avi files with xvid codec (packed / unpacked)?
 

Subject: [PATCH] dvbvideosink: add profile to caps and ignore dummy packed
 B-Frames


diff --git a/gstdvbvideosink.c b/gstdvbvideosink.c
index d8d5635..37b3ac0 100755
--- a/gstdvbvideosink.c
+++ b/gstdvbvideosink.c
@@ -176,12 +176,14 @@ GST_STATIC_PAD_TEMPLATE (
 	GST_PAD_SINK,
 	GST_PAD_ALWAYS,
 	GST_STATIC_CAPS (
-	"video/mpeg, "
 #ifdef HAVE_MPEG4
-		"mpegversion = (int) { 1, 2, 4 }, "
-#else
-		"mpegversion = (int) { 1, 2 }, "
+	"video/mpeg, "
+		"mpegversion = (int) 4, "
+		"profile = (string) { simple, advanced-simple }, "
+		VIDEO_CAPS "; "
 #endif
+	"video/mpeg, "
+		"mpegversion = (int) { 1, 2 }, "
 		VIDEO_CAPS "; "
 #ifdef HAVE_H264
 	"video/x-h264, "
@@ -815,6 +817,9 @@ static GstFlowReturn gst_dvbvideosink_render(GstBaseSink *sink, GstBuffer *buffe
 		}
 	}
 #endif
+	/* remove dummy packed B-Frame */
+	if (self->codec_type == CT_MPEG4_PART2 && data_len <= 7)
+		goto ok;
 
 	pes_header[0] = 0;
 	pes_header[1] = 0;
-- 

PS. Try without DTS & PTS patch in PES header

Attached Files

•  0001-dvbvideosink-add-profile-to-caps-and-ignore-dummy-pa.patch   1.08KB   13 downloads