203 replies to this topic

[pieterg]

But the memleak fix (according to the description) is part of the utf-8 truncate commit?

[awx]

But the memleak fix (according to the description) is part of the utf-8 truncate commit?

Patch for the memleak alone was posted earlier

[awx]

But the memleak fix (according to the description) is part of the utf-8 truncate commit?

But you could take the entire patch, and just remove the if that does the truncating,
this would fix the titles in more cases, and not truncate the text. It would only leave the posiblity that some titles and text still wouldnt be displayed but wouldnt have the problem you are concerned about

I would recommend that, at least it would fix the situation for some more cases

Edited by awx, 23 January 2012 - 11:52.

[DimitarCC]

I'd rather not commit something before it's final

Has it been investigated what happens if an incomplete utf8 sequence it present, because of the truncate hack?
We have to know how the various utf8 parsers (c++, python) handle such situations.
Or rather, we should never store incomplete utf8 sequences.

By using a truncate function such as this (untested):

   	 if (data[length - 1] > 0x7F)
		{
			do
			{
				/* remove all UTF data bytes, not including the start byte, which is {0xC0 <= startbyte <= 0xFD} */
				length--;
			}
			while (length > 0 && data[length - 1] <= 0xBF); /* remove only databytes */
		}
		/* remove the UTF startbyte, or normal ascii character */
		if (length > 0)
		{
			length--;
		}
		data[length] = 0;

Well we was see some commit in the past that in some situations was even broke the image. I cant understand why not commiting this! I dont think so is good ided to cant read EPG instead to put some untested function in unstable image and users to can test it is it work instead just some people to can do it! I think more real testers - more chance bug to be found ( or at least to understand will the commit have side effect)!

[pieterg]

But the memleak fix (according to the description) is part of the utf-8 truncate commit?

But you could take the entire patch, and just remove the if that does the truncating,
this would fix the titles in more cases, and not truncate the text. It would only leave the posiblity that some titles and text still wouldnt be displayed but wouldnt have the problem you are concerned about

I think that would effectively truncate the utf8 as well, wouldn't it?
Or does it produce a short event descriptor with invalid length?

I'd say we fix the truncate, and commit the result.
Or if you prefer, I'll make a utf8 truncate routine. But not right now, I'm working.

[awx]

But the memleak fix (according to the description) is part of the utf-8 truncate commit?

But you could take the entire patch, and just remove the if that does the truncating,
this would fix the titles in more cases, and not truncate the text. It would only leave the posiblity that some titles and text still wouldnt be displayed but wouldnt have the problem you are concerned about

I think that would effectively truncate the utf8 as well, wouldn't it?
Or does it produce a short event descriptor with invalid length?

I'd say we fix the truncate, and commit the result.
Or if you prefer, I'll make a utf8 truncate routine. But not right now, I'm working.

With out the if check, would produce the same affect as now, the length would roll over to the higher bits and would be 0 and never processed.
However since we split the two poritions in my change we have a better chance of not rolling over.

I will rewirte some of the event data processing to store titles seperatly, during this week and have a proper solution to this.

[awx]

But the memleak fix (according to the description) is part of the utf-8 truncate commit?

But you could take the entire patch, and just remove the if that does the truncating,
this would fix the titles in more cases, and not truncate the text. It would only leave the posiblity that some titles and text still wouldnt be displayed but wouldnt have the problem you are concerned about

I think that would effectively truncate the utf8 as well, wouldn't it?
Or does it produce a short event descriptor with invalid length?

I'd say we fix the truncate, and commit the result.
Or if you prefer, I'll make a utf8 truncate routine. But not right now, I'm working.

I would not make the truncate routine, as I want to fix this so that we dont have to truncate

[ims]

I would not make the truncate routine, as I want to fix this so that we dont have to truncate

it would be nice :-)

[DimitarCC]

I think the temporary fix should be commited and if needed then reverted or changed!!!!

[bigroma]

In first look all works great now.
Thx guys.

[awx]

Here is a patch based on the current PLi source
This changes the way data is stored in the epgcache, it should solve all title trimming, but is a large change and needs lots of testing

Attached Files

•  restructure.patch.tar.gz   6.56KB   12 downloads

[ims]

apply and it seems, it works without problems (after one gdos)...

[bigroma]

Last patch incompatible with "enigma2-plugin-systemplugins-crossepg - 0.6.2+git284" when i load epg from net i had gsod from CrossEPG_Loader

It stopped on loading /media/hdd/crossepg/ext.epg.dat

I'd get epg from Linuxsat ExUSSR xmltv provider.

Edited by bigroma, 24 January 2012 - 23:56.

[awx]

Last patch incompatible with "enigma2-plugin-systemplugins-crossepg - 0.6.2+git284" when i load epg from net i had gsod from CrossEPG_Loader

It stopped on loading /media/hdd/crossepg/ext.epg.dat

I'd get epg from Linuxsat ExUSSR xmltv provider.

Since we change the format of how internals are stored you can not use the old .dat file you have to start fresh

[awx]

This patch will likely break stroed recordings EIT data parsing
Not sure if the time of the box is updated properly on full reboot, could someone check that?

[daddelfalk]

Hi,

time is grabbed from DVB-Transmission for Default E2-configuration...

[awx]

Hi,

time is grabbed from DVB-Transmission for Default E2-configuration...

Thanks, then my patch will hopefully not affect this.

[bigroma]

I understood this. I'm just pointing you to problem and waiting for it resolve.
I think bettter add convert ext.epg.dat to CrossEPG for avoiding different format external epg file.

[awx]

For the current patch, I plan to move the eit write portion into eServiceEvent as this is the palce that the reading of the file also happens.
This will make it a bit easier to control the changes to them both and create one consistent format and independant of the epgCache format to decouple this portion which will allow us to have a bit more freedom to change the epg format.

I am not sure where an external epg is loaded for something like cross epg, so I will have to look into that. That being said the side affects of changing the data format are that previous dat may no longer be compatible.
Which is why data that must persist like eit recording info will be made into a format that will hopefully never change and if it does will be somehow backwards compatible.
As for epgcache I will try to do the same thing as well so that at least the saved format will be compatible so that it can be loaded, but I cant guarantee this

[awx]

There is a small bug in the eventData desctructor
CacheSize -= it->second.second[1];
Should be
CacheSize -= (it->second.second[1] -2);

It did not remove the count of the header data for each descriptor