203 replies to this topic

[ims]

I saw into epg.dat. There is text still in "Two Char Mapping" and missing same parts, as on screen.

[DimitarCC]

Well it seemd there is no fix for that yet! Right? On my EPG many many titles get missing by this bug! So i hope soon fix!

[awx]

Well it seemd there is no fix for that yet! Right? On my EPG many many titles get missing by this bug! So i hope soon fix!

I think I have found the problem and am working on a fix for this

[bigroma]

Great, we're waiting patch for testing.

[awx]

Great, we're waiting patch for testing.

Here is the fix or potential fix if you will
@ims - you were right it had to do with the length but not in the way I had originally thought

Changes
------------------

Fix for short event description data not showing up

After converting our titles to UTF8 we have the problem that they no
longer fit into a short event descriptor structure because the length
is only 8 bits. Split the title and text of short event descriptor to
better deal with this. Also add a hack to hardcode a data length when
our data is too long.

Splitting the title and text makes us waste some memory, however
since we split title and text we end up saving anywhere in the range
of 10 to 30 % on title storage. This is because previously we checked
to see if the entire descriptor was a duplicate, which didnt match duplicate
titles when it could have.

Also fix a memory leak in the process.

TODO: fix all text storage not to use the dvb descriptor format, possibly use
a title and text index as this is sort of what we are doing now with the descriptor
map.
------------------

Use only one constructor for eventData

We can use one eventData constructor as they both do more or less the same thing.

Attached Files

•  shortdescfix.tar.gz   3.45KB   17 downloads

[ims]

it is patch for OpenPLi git ?

[awx]

it is patch for OpenPLi git ?

Yes, it should be, I hadn't pull the last 4 commits, but these files were not affect in those, so it should merge right in.
Although I had merged my changes on to master from a branch I had, which I may have messed up, but diff shows that the areas affect are the same, so should be fine.

[ims]

thanks, it seems, it works.

[awx]

thanks, it seems, it works.

thanks as well to you for all the help

[Nikei]

Today has updated, too most

Attached Files

•  screenshot.jpg   72.73KB   32 downloads

[DimitarCC]

so is that will be commited to ofitial repository?

[bigroma]

Thx, but after patch without ANY changes:)

36e 9e

Maybe you see this sats or maybe some information from me can help?

Edited by bigroma, 23 January 2012 - 07:50.

[awx]

Thx, but after patch without ANY changes:)

36e 9e

Maybe you see this sats or maybe some information from me can help?

Will likely need to make a special patch for you to get some more information.
In the meantime there are two added in the patch with 255 - 6, could you change these to 255 - 7

Little explanation behind this, 255 - 6 is a check to make sure that when we add our title length together with the overhead of the rest of the descriptor the descriptor length still fits into the byte used to store the length. Now I thought that 255 -6 would end up storing 0xFF in the field when our data was too long.

So if you could try 255 - 7, and let me know that would be great. Otherwise I will try and provide you with a patch for more debug later in the day.

[blzr]

seems that it hasn't been commited yet, that's why problem still appears...
http://openpli.git.s...igma2;a=summary

[awx]

Please let me know if the results are based on a local patched build or from the nightly.

[bigroma]

Excuse me, but i didn't understand where i must change 255-6 to 256-7. I didn't see it in last patch.

[ims]

I dont know, why it was not commited. ... was weekend

Edited by ims, 23 January 2012 - 09:17.

[pieterg]

I'd rather not commit something before it's final

Has it been investigated what happens if an incomplete utf8 sequence it present, because of the truncate hack?
We have to know how the various utf8 parsers (c++, python) handle such situations.
Or rather, we should never store incomplete utf8 sequences.

By using a truncate function such as this (untested):

   	 if (data[length - 1] > 0x7F)
        {
            do
            {
                /* remove all UTF data bytes, not including the start byte, which is {0xC0 <= startbyte <= 0xFD} */
                length--;
            }
            while (length > 0 && data[length - 1] <= 0xBF); /* remove only databytes */
        }
        /* remove the UTF startbyte, or normal ascii character */
        if (length > 0)
        {
            length--;
        }
        data[length] = 0;

[Nikei]

This problem has appeared after updating on January, 17th, before all was good.

[awx]

I'd rather not commit something before it's final

Has it been investigated what happens if an incomplete utf8 sequence it present, because of the truncate hack?
We have to know how the various utf8 parsers (c++, python) handle such situations.
Or rather, we should never store incomplete utf8 sequences.

By using a truncate function such as this (untested):

   	 if (data[length - 1] > 0x7F)
		{
			do
			{
				/* remove all UTF data bytes, not including the start byte, which is {0xC0 <= startbyte <= 0xFD} */
				length--;
			}
			while (length > 0 && data[length - 1] <= 0xBF); /* remove only databytes */
		}
		/* remove the UTF startbyte, or normal ascii character */
		if (length > 0)
		{
			length--;
		}
		data[length] = 0;

If you dont take this fix, then you should at least for the time being take the memleak patch I have posted.
A full solution to this fix is only possible with a large rewrite of eventData, or some other more hacked methods, as such this is not a quick fix and the changes should be reverted.