128 replies to this topic

[doglover]

It still says:

root@et9x00:~# yafc                                                           
yafc 1.3.5                                                                    
This program comes with ABSOLUTELY NO WARRANTY; for details type 'warranty'.  
This is free software; type 'copyright' for details.                          
                                                                              
yafc>

And they are not skipping anything anymore. Yafc just uploads everything

 

put -f -n *.gz

and

put --force --newer *.gz

 

Have the same result.  Upload everything.  And i am certain I only changed only 1 file of the 39 to be uploaded.

Transferred 39 files, total size 1996157318 6iB.

Willy

[athoik]

As @sebastinas says on the bug request it is better to create debug/verbose log that will help diagnosing the issue.

Remove any sensitive information (password/ip/hostname).

[LraiZer]

in latest version maybe try -F or --force-newer (do not use cached information with --newer)

 

put -F *.gz

put --force-newer *.gz

[LraiZer]

on original yafc 1.3.5, have you tried without adding the -f, --force, parm on your put command?

put -np *.gz

[doglover]

on original yafc 1.3.5, have you tried without adding the -f, --force, parm on your put command?

put -np *.gz

 

No I did not try this.

But now I did.

With no files updated, I did not get any uploads (As it should)

Updated one file, and Bingo                                                                         

Transferred 1 files, Skipped 38 files, total size 2002416006 hiB.

And no confirmation asked.  I thought I needed the force options to avoid confirmations.  I am running this automated (unmanned) and in the documentation you have this:

‘-f’
‘--force’
    Overwrite existing destinations, never prompt.

Will continue on this path

 

Willy

[doglover]

in latest version maybe try -F or --force-newer (do not use cached information with --newer)

 

put -F *.gz

put --force-newer *.gz

 

Result:

yafc url:/epg_data> put --force-newer *.gz      
Remote file 'rytecxmltvxxxxxxx.gz' exists                         
Local: 1988933 bytes, Sat Apr 25 03:58:14 2015                     
Remote: 1988933 bytes, Sat Apr 25 09:32:04 2015                    
Overwrite? [yncauR, ? for help] c

Request confirmation!!!!

 

Willy

Edited by doglover, 25 April 2015 - 08:35.

[doglover]

Got the switches now working for the FTP servers.
I think everything hinged on the misconception of the -f (force) option.

However, I have one sftp server which does not behave.

yafc: 'put -n rytec.channels*.gz'
** start parsing directory listing ***
-rw-r--r-- 1 1000 1004 59722 Apr 26 07:02 rytec.channels.xml-sat.gz
-rwxr--r-- 1 1000 1002 2232 Apr 26 06:54 rytec.channels-greece-en.xml.gz
drwxr-xr-x 2 1000 1004 4096 Apr 26 07:12 .
-rw-r--r-- 1 1000 1004 137025 Apr 17 00:20-rwxr--r-- 1 1000 1002 7248 Apr 26 06:54 rytec.channels-denmark.xml.gz
-rwxr--r-- 1 1000 1002 8933 Apr 26 06:54 rytec.channels-norway.xml.gz
-rw-r--r-- 1 1000 1002 9400 Apr 26 06:54 rytec.channels-sweden.xml.gz
-rwxr-xr-x 1 0 0 41 Apr 10 12:07 index.php
-rw-r--r-- 1 1000 1004 107324 Apr 26 07:02 rytec.channels.xml.gz
-rwxr-xr-x 1 1000 1004 63858 Apr 26 06:54 rytec.channels-sat.xml.gz
drwxr-xr-x 3 0 0 4096 Apr 10 11:41 ..
-rw-r--r-- 1 1000 1004 10172 Apr 26 06:54 rytec.channels-slovenia.xml.gz
-rwxr--r-- 1 1000 1002 10863 Apr 26 06:54 rytec.channels-finland.xml.gz
-rwxr-xr-x 1 1000 1004 1354 Apr 26 06:54 rytec.channels-caledonia.xml.gz
-rwxr--r-- 1 1000 1002 47497 Apr 26 07:02 rytec.channels.xml-cable.gz
*** end parsing directory listing ***
added directory '/epg_data' to cache
marked directory '/epg_data' for flush
error flushing directory '�ˆF' (not cached)

And all these files are uploaded. But the ones uploaded are older than these on the server, They should be skipped.
Am I missing something?

Willy

Edited by doglover, 28 April 2015 - 14:09.

[doglover]

Got something cooked up.

Request the local  and remote filelists.  And compare them on filesize.  (Date is a problem).  If they are different I copy the files to an empty dir.  And later upload this dir.

#!/bin/sh

rm /hdd/epg/tobeuploaded/*.*

yafc -q <<EOF
open Westland
lcd /hdd/epg/
get -f -n -p *.gz
close
exit
EOF

yafc -q <<EOF3
open kingserver
lcd /media/hdd/epg/
ls -l *.gz > /media/hdd/epg/remote.dir
close
exit
EOF3

cd /hdd/epg/
ls -le *.gz > /media/hdd/epg/local.dir
LOCALFILE=/hdd/epg/local.dir
REMOTE=/hdd/epg/remote.dir
while read line;do
  LengteLocal=${line:32:11}
  FileName=${line:69:40}
  RemoteLine=$(grep -e "$FileName" $REMOTE)
  LengteRemote=${RemoteLine:30:11}
  if [ $LengteLocal -ne $LengteRemote ] ; then
      cp -f /hdd/epg/$FileName  /hdd/epg/tobeuploaded/$FileName
  fi
done < $LOCALFILE

yafc -q <<EOF2
lcd /hdd/epg/
open WoS
put -np *.gz
close
open VUplus
put -np *.gz
close
lcd /hdd/epg/tobeuploaded
open kingserver
put --force *.gz
close
exit
EOF2

Now I use 2 temp files for storing the localdir and remotedir lists.

I want to do away with these 2 temp files.

Can I do this directly in the bash file?  Using a variable?

Problem is of course one of the dirlists is requested using yafc.  And storing to a variabel is not working.  Or I do not see it.

 

Willy

Edited by doglover, 28 April 2015 - 14:19.

[LraiZer]

New in 1.3.6
------------

* Fix shortpath calculation. Thanks to Celelibi for the patch. (#48)
* Fix %u in prompts if built without libssh. (#49)
* Fix multiple memory leaks, crashes and use-after-free errors.
* Use cached information if available on get/put --newer. Thanks to Celelibi for
  the patch.
* Fix put/get -n switch for sftp connections. (#53)
* Fix libssh deprecation and bump version requirements accordingly.

https://github.com/s...ases/tag/v1.3.6

Attached Files

•  yafc_1.3.6-r0_mips32el.ipk   106.5KB   7 downloads

[doglover]

Any chance to add this plug-in to the feed?

The bitbake recipe (if adjusted for the version) published above works.  (I used it myself).

 

Of course not many users of an enigma2 receiver will ever need this.  But the ones who need a good FTP program, have a good tool on this application.

At moment yafc is used to update the EPG servers.  And it does its job without any problem.

 

Willy

Edited by doglover, 16 May 2015 - 07:02.

[doglover]

I switched to an hd2400 and OpenPLi6.0 and I have problems running yafc now.

 

I get this:

root@hd2400:~# yafc                                                           
yafc: /usr/lib/libcrypto.so.1.0.0: version `OPENSSL_1.0.0' not found (required

When i tried:

root@hd2400:~# opkg install openssl                           
Package openssl (1.0.2j-r0.0) installed in root is up to date.

How can I get around this problem?

[WanWizard]

Recompile yafc for the correct environment?

 

From an earlier thread I understood that you had a bitbake recipe for creating it, was that ever added to our build environment? That would perhaps be a more permanent solution.

[doglover]

I can compile it but it wont install anymore.  All kind of dependencies are not fulfilled anymore.

I have to use the old ipk to install.  But on OpenPLi it installs but wont run.

So my question can I do something on that error from above, so it will run again.

 

Otherwise I will have to rewrite the routine for uploading the EPG using curl.

Problem I have to resolve is how to pass on passwords in curl containing all kinds of special characters.

 

Bummer.

[WanWizard]

That is beyond my skills I'm afraid.

[athoik]

@doglover, How do you compile the yafc?

The recipes always take of the depends. I can recompile it and attach it too, and if that works commit to OpenPLi feeds, so no need to worry again in the future.

[doglover]

Compile it with the bitbake recipe published somewhere above in the OpenPLi build environment on Ubuntu.

But there ends my knowledge as well.

 

Found a FTPclient (command line - lftp) for installation on the NAS.  Will study that and see if I can do the work on the NAS. 

 

However if you can help with a version that works on OpenPLi6.0 very much appreciated.  It would make my live easier.

[blzr]

FWIW, just compiled yafc using MiLo's recipe:
https://forums.openp...ndpost&p=753559
for arm amd mipsel (on the develop branch, though)

built fine, installed ok (I tried arm package on vs1500)
you can try attached mips32el version - should (?) work on 6.0 release...

Attached Files

•  yafc_1.3.7+git806+4321cd3-r0.0_mips32el.ipk   111.64KB   4 downloads

[WanWizard]

If that works, can you add the recipe to develop?

[blzr]

no problem, if someone confirms it (really) works as supposed...

for now, as I said, builds ok, installs ok and:

root@vs1500:~# yafc 
yafc 1.3.7
This program comes with ABSOLUTELY NO WARRANTY; for details type 'warranty'.
This is free software; type 'copyright' for details.

yafc> 

[doglover]

root@hd2400:~# opkg install /tmp/yafc_1.3.7+git806+4321cd3-r0.0_mips32el.ipk  
Upgrading yafc from 1.3.5-r0 to 1.3.7+git806+4321cd3-r0.0 on root.            
Collected errors:                                                             
* calculate_dependencies_for: Cannot satisfy the following dependencies for ya
c:                                                                            
*      libssh (>= 0.7.3) *                                                   
* opkg_solver_install: Cannot install package yafc.                          
root@hd2400:~#

BTW i have another problem on this HD2400.  Suddenly I cannot access it anymore via Windows explorer or via FTP

DCC is OK, so is Putty.

 

Willy