Jump to content


Photo

FTP Client (not as a plugin)

ET6000

  • Please log in to reply
128 replies to this topic

Re: FTP Client (not as a plugin) #21 doglover

  • Rytec EPG Team
  • 16,974 posts

+635
Excellent

Posted 16 April 2015 - 10:26

Let me explain my problem.

 

I grab and upload the EPG data to the distribution servers.  I have 2 PC's for doing this.  On at my house, and one at Rytec's.

The one at rytec's is connected to the internet with a crappy internet line. Upload 0.5 Mbit/s

Historically this PC has been used to distribute all the files.  Which really took a long time.  However my own internet line was not much better at an Upload of 1 Mbit/s

So we kept suffering.

 

Now very recently my home line has been upgraded. (Proximus performed some upgrades on the infrastructure)  Now my line says Upload of 2 mbit/s.  However I see speeds in excess of 3 mbit/s in practice.  This varies however from test to test.  The line does not seem to be capped in anyway.

 

So I am trying now to distribute everything from my home.  But I do not want my PC's to run all day.

One of my linux boxes is running all the time.  And I am investigating the possibility to do the distribution with this linux machine.

 

Therefore I need a command line FTP program for the linux OpenPli box, which I can automate.

So far the answers where not very succesfull.

 

Willy

 

PS: On my windows machines I am using Filezilla, WinSCP and CoreFTP. 

Filezilla does not have an commandline interface.  CoreFTP is a crappy program, and WinSCP works good on most servers, but there is one server which it does not like.  The full program works OK, but in the command line version, I get continously disconnects during an upload to this one server.


Edited by doglover, 16 April 2015 - 10:27.

~~Rytec Team~~
Maxytec Multibox SE OpenPli (used as mediaplayer)
Mutant HD2400 OpenPli
Vu+ Duo OpenPli (backup)

Synology NAS

Sat: 13E, 19.2E, 23.5E and 28.2E
*Pli/Rytec EPG POWERED*


Re: FTP Client (not as a plugin) #22 betacentauri

  • PLi® Core member
  • 7,185 posts

+323
Excellent

Posted 16 April 2015 - 17:00

For one box or for all?
For one box download latest busy box binary and put it on the box as extra binary or replace the original. Well better at first put it to /tmp, make it executable and test it.
http://www.busybox.n...inaries/latest/

 

Why not use this?? It works:

root@et7x00:~# wget http://www.busybox.net/downloads/binaries/latest/busybox-mipsel
Connecting to www.busybox.net (140.211.167.224:80)
busybox-mipsel       100% |********************************************************|  1539k  0:00:00 ETA
root@et7x00:~# chmod 755 busybox-mipsel 
root@et7x00:~# ./busybox-mipsel 
BusyBox v1.21.1 (2013-07-08 11:09:23 CDT) multi-call binary.
BusyBox is copyrighted by many authors between 1998-2012.
Licensed under GPLv2. See source distribution for detailed
copyright notices.

Usage: busybox [function [arguments]...]
   or: busybox --list[-full]
   or: busybox --install [-s] [DIR]
   or: function [arguments]...

        BusyBox is a multi-call binary that combines many common Unix
        utilities into a single executable.  Most people will create a
        link to busybox for each function they wish to use and BusyBox
        will act like whatever it was invoked as.

Currently defined functions:
        [, [[, acpid, add-shell, addgroup, adduser, adjtimex, arp, arping, ash, awk, base64, basename,
        beep, blkid, blockdev, bootchartd, brctl, bunzip2, bzcat, bzip2, cal, cat, catv, chat, chattr,
        chgrp, chmod, chown, chpasswd, chpst, chroot, chrt, chvt, cksum, clear, cmp, comm, conspy, cp,
        cpio, crond, crontab, cryptpw, cttyhack, cut, date, dc, dd, deallocvt, delgroup, deluser,
        depmod, devmem, df, dhcprelay, diff, dirname, dmesg, dnsd, dnsdomainname, dos2unix, du,
        dumpkmap, dumpleases, echo, ed, egrep, eject, env, envdir, envuidgid, ether-wake, expand, expr,
        fakeidentd, false, fbset, fbsplash, fdflush, fdformat, fdisk, fgconsole, fgrep, find, findfs,
        flock, fold, free, freeramdisk, fsck, fsck.minix, fsync, ftpd, ftpget, ftpput, fuser, getopt,
        getty, grep, groups, gunzip, gzip, halt, hd, hdparm, head, hexdump, hostid, hostname, httpd,
        hush, hwclock, id, ifconfig, ifdown, ifenslave, ifplugd, ifup, inetd, init, insmod, install,
        ionice, iostat, ip, ipaddr, ipcalc, ipcrm, ipcs, iplink, iproute, iprule, iptunnel, kbd_mode,
        kill, killall, killall5, klogd, last, less, linux32, linux64, linuxrc, ln, loadfont, loadkmap,
        logger, login, logname, logread, losetup, lpd, lpq, lpr, ls, lsattr, lsmod, lsof, lspci, lsusb,
        lzcat, lzma, lzop, lzopcat, makedevs, makemime, man, md5sum, mdev, mesg, microcom, mkdir,
        mkdosfs, mke2fs, mkfifo, mkfs.ext2, mkfs.minix, mkfs.vfat, mknod, mkpasswd, mkswap, mktemp,
        modinfo, modprobe, more, mount, mountpoint, mpstat, mt, mv, nameif, nanddump, nandwrite,
        nbd-client, nc, netstat, nice, nmeter, nohup, nslookup, ntpd, od, openvt, passwd, patch, pgrep,
        pidof, ping, ping6, pipe_progress, pivot_root, pkill, pmap, popmaildir, poweroff, powertop,
        printenv, printf, ps, pscan, pstree, pwd, pwdx, raidautorun, rdate, rdev, readahead, readlink,
        readprofile, realpath, reboot, reformime, remove-shell, renice, reset, resize, rev, rm, rmdir,
        rmmod, route, rpm, rpm2cpio, rtcwake, run-parts, runlevel, runsv, runsvdir, rx, script,
        scriptreplay, sed, sendmail, seq, setarch, setconsole, setfont, setkeycodes, setlogcons,
        setserial, setsid, setuidgid, sh, sha1sum, sha256sum, sha3sum, sha512sum, showkey, slattach,
        sleep, smemcap, softlimit, sort, split, start-stop-daemon, stat, strings, stty, su, sulogin,
        sum, sv, svlogd, swapoff, swapon, switch_root, sync, sysctl, syslogd, tac, tail, tar, tcpsvd,
        tee, telnet, telnetd, test, tftp, tftpd, time, timeout, top, touch, tr, traceroute, traceroute6,
        true, tty, ttysize, tunctl, udhcpc, udhcpd, udpsvd, umount, uname, unexpand, uniq, unix2dos,
        unlzma, unlzop, unxz, unzip, uptime, users, usleep, uudecode, uuencode, vconfig, vi, vlock,
        volname, wall, watch, watchdog, wc, wget, which, who, whoami, whois, xargs, xz, xzcat, yes,
        zcat, zcip

root@et7x00:~# ./busybox-mipsel ftpput
BusyBox v1.21.1 (2013-07-08 11:09:23 CDT) multi-call binary.

Usage: ftpput [OPTIONS] HOST [REMOTE_FILE] LOCAL_FILE

Upload a file to a FTP server

        -v,--verbose            Verbose
        -u,--username USER      Username
        -p,--password PASS      Password
        -P,--port NUM           Port

root@et7x00:~# 

Xtrend ET-9200, ET-8000, ET-10000, OpenPliPC on Ubuntu 12.04

Re: FTP Client (not as a plugin) #23 LraiZer

  • Senior Member
  • 101 posts

+19
Neutral

Posted 16 April 2015 - 21:42

Yafc was the only command line ftp that worked to solve continous disconnects during uploads to a server in an automated script on my ubuntu build machine.

 



Re: FTP Client (not as a plugin) #24 doglover

  • Rytec EPG Team
  • 16,974 posts

+635
Excellent

Posted 17 April 2015 - 08:30

I will try the busybox.  (already downloaded it)  Did not know a mipsel variant existed.  Thought it had to come from the image buidelrs.

 

Unfortunally Yafc is not compiled for mipsel.  And I do not know enough about compiling it myself.

If some-one can do this.  I will try it.

 

Willy


~~Rytec Team~~
Maxytec Multibox SE OpenPli (used as mediaplayer)
Mutant HD2400 OpenPli
Vu+ Duo OpenPli (backup)

Synology NAS

Sat: 13E, 19.2E, 23.5E and 28.2E
*Pli/Rytec EPG POWERED*


Re: FTP Client (not as a plugin) #25 doglover

  • Rytec EPG Team
  • 16,974 posts

+635
Excellent

Posted 17 April 2015 - 13:11

The ftpput (and get) does not do wildcards. So I had to resort to writing a .sh file for doing the job.

However on hindsight this is not a bad thing, as I have much more control on what is going on.

 

To give you an idea about the .sh file:

#!/bin/sh
date
k=1
FILE=/hdd/epg/Filelist.txt
while read line;do
        echo "Line # $k: $line"
        if k < 17 then
            /hdd/epg/busybox-mipsel ftpget -v --user=xxxxxxx --pass=xxxxxxx IP-adres /hdd/epg/$line /ready/$line
        fi
        /hdd/epg/busybox-mipsel ftpput -v --user=xxxxxxx --pass=xxxxxxx1 IP-adres1 /$line /hdd/epg/$line
        /hdd/epg/busybox-mipsel ftpput -v --user=xxxxxxx --pass=xxxxxxx2 IP-adres2 /$line /hdd/epg/$line
        /hdd/epg/busybox-mipsel ftpput -v --user=xxxxxxx --pass=xxxxxxx3 IP-adres3 /$line  /hdd/epg/$line       
        let k=k+1
done < $FILE
date

Executing this, and piping the output to a file, gives me also a nice log file.

 

Willy

 

PS:  the if loop is in there because I do not need to download all the files (only part)


Edited by doglover, 17 April 2015 - 13:21.

~~Rytec Team~~
Maxytec Multibox SE OpenPli (used as mediaplayer)
Mutant HD2400 OpenPli
Vu+ Duo OpenPli (backup)

Synology NAS

Sat: 13E, 19.2E, 23.5E and 28.2E
*Pli/Rytec EPG POWERED*


Re: FTP Client (not as a plugin) #26 doglover

  • Rytec EPG Team
  • 16,974 posts

+635
Excellent

Posted 17 April 2015 - 13:26

Left to done:  one of the servers I can only log into by SFTP.  So find an SFTP application for a mipsel architecture.

Or find somebody which can compile Yafc for me.  (inclusive of the shared libraries)

 

Willy


Edited by doglover, 17 April 2015 - 13:53.

~~Rytec Team~~
Maxytec Multibox SE OpenPli (used as mediaplayer)
Mutant HD2400 OpenPli
Vu+ Duo OpenPli (backup)

Synology NAS

Sat: 13E, 19.2E, 23.5E and 28.2E
*Pli/Rytec EPG POWERED*


Re: FTP Client (not as a plugin) #27 theparasol

  • Senior Member
  • 4,157 posts

+198
Excellent

Posted 17 April 2015 - 14:51

I just learned about curl while doing some searching for sftp connections.

What even is better: Its standard in openpli images!

Perhaps its a better idea to start using curl as its supports sftp too :)

 

You can use it like this:

 

curl -T uploadfile -u user:passwd ftp://ftp.upload.com/myfile

 

Read more here: http://curl.haxx.se/docs/manual.html


@Camping: ZGemma H.2S, Technisat Multytenne 4-in-1 @Home: Edision Mini 4K, Wave Frontier T55, EMP Centauri EMP DiSEqC 8/1 switch, 4x Inverto Ultra Black single LNB


Re: FTP Client (not as a plugin) #28 athoik

  • PLi® Core member
  • 8,458 posts

+327
Excellent

Posted 17 April 2015 - 15:23

Left to done: one of the servers I can only log into by SFTP. So find an SFTP application for a mipsel architecture.
Or find somebody which can compile Yafc for me. (inclusive of the shared libraries)

Willy


Did you try scp?
Wavefield T90: 0.8W - 1.9E - 4.8E - 13E - 16E - 19.2E - 23.5E - 26E - 33E - 39E - 42E - 45E on EMP Centauri DiseqC 16/1
Unamed: 13E Quattro - 9E Quattro on IKUSI MS-0916

Re: FTP Client (not as a plugin) #29 doglover

  • Rytec EPG Team
  • 16,974 posts

+635
Excellent

Posted 17 April 2015 - 16:32

It is not in OpenPLi, but it is in the feed.

So you have to install it.

 

But: curl: (1) Protocol "sftp" not supported or disabled in libcurl

 

Willy                                                   

                                                     
                                                      


~~Rytec Team~~
Maxytec Multibox SE OpenPli (used as mediaplayer)
Mutant HD2400 OpenPli
Vu+ Duo OpenPli (backup)

Synology NAS

Sat: 13E, 19.2E, 23.5E and 28.2E
*Pli/Rytec EPG POWERED*


Re: FTP Client (not as a plugin) #30 doglover

  • Rytec EPG Team
  • 16,974 posts

+635
Excellent

Posted 17 April 2015 - 16:33

 

Left to done: one of the servers I can only log into by SFTP. So find an SFTP application for a mipsel architecture.
Or find somebody which can compile Yafc for me. (inclusive of the shared libraries)

Willy


Did you try scp?

 

 

yes: problems with using logins user/password and login key

 

Willy


~~Rytec Team~~
Maxytec Multibox SE OpenPli (used as mediaplayer)
Mutant HD2400 OpenPli
Vu+ Duo OpenPli (backup)

Synology NAS

Sat: 13E, 19.2E, 23.5E and 28.2E
*Pli/Rytec EPG POWERED*


Re: FTP Client (not as a plugin) #31 theparasol

  • Senior Member
  • 4,157 posts

+198
Excellent

Posted 17 April 2015 - 17:16

But: curl: (1) Protocol "sftp" not supported or disabled in libcurl                                                   

                                                      

 

Yes, indeed in the feed.... what a bummer!

 

But perhaps a dev is willing to add support for it:

 

http://iamnearlyther...lities-to-curl/

 

It can work, only packages in feed have to be compiled differently.


@Camping: ZGemma H.2S, Technisat Multytenne 4-in-1 @Home: Edision Mini 4K, Wave Frontier T55, EMP Centauri EMP DiSEqC 8/1 switch, 4x Inverto Ultra Black single LNB


Re: FTP Client (not as a plugin) #32 doglover

  • Rytec EPG Team
  • 16,974 posts

+635
Excellent

Posted 17 April 2015 - 17:36

SCP:  Can you give me an example to upload a file on a sftp server with username, password en rsa-key

And not the printout of the help file:

 

usage: scp [-1246BCpqrv] [-c cipher] [-F ssh_config] [-i identity_file]     
           [-l limit] [-P port] [-S program]                                
           [[user@]host1:]file1 [...] [[user@]host2:]file2     

 

I can get this myself.  I only cannot seems to figure out how to construct the command line.

 

Willy          


~~Rytec Team~~
Maxytec Multibox SE OpenPli (used as mediaplayer)
Mutant HD2400 OpenPli
Vu+ Duo OpenPli (backup)

Synology NAS

Sat: 13E, 19.2E, 23.5E and 28.2E
*Pli/Rytec EPG POWERED*


Re: FTP Client (not as a plugin) #33 doglover

  • Rytec EPG Team
  • 16,974 posts

+635
Excellent

Posted 17 April 2015 - 17:38

 

But: curl: (1) Protocol "sftp" not supported or disabled in libcurl                                                   

                                                      

 

Yes, indeed in the feed.... what a bummer!

 

But perhaps a dev is willing to add support for it:

 

http://iamnearlyther...lities-to-curl/

 

It can work, only packages in feed have to be compiled differently.

 

 

This should be helpfull.

 

Willy


~~Rytec Team~~
Maxytec Multibox SE OpenPli (used as mediaplayer)
Mutant HD2400 OpenPli
Vu+ Duo OpenPli (backup)

Synology NAS

Sat: 13E, 19.2E, 23.5E and 28.2E
*Pli/Rytec EPG POWERED*


Re: FTP Client (not as a plugin) #34 betacentauri

  • PLi® Core member
  • 7,185 posts

+323
Excellent

Posted 17 April 2015 - 18:09

Afaik you cannot use scp for a sftp server. You can only use scp when you can login via ssh.

 

But you can install sftp command:

root@et7x00:~# opkg install openssh-sftp
Installing openssh-sftp (6.7p1-r0.0) on root.
root@et7x00:~# sftp
usage: sftp [-1246aCfpqrv] [-B buffer_size] [-b batchfile] [-c cipher]
          [-D sftp_server_path] [-F ssh_config] [-i identity_file] [-l limit]
          [-o ssh_option] [-P port] [-R num_requests] [-S program]
          [-s subsystem | sftp_server] host
       sftp [user@]host[:file ...]
       sftp [user@]host[:dir[/]]
       sftp -b batchfile [user@]host


Edited by betacentauri, 17 April 2015 - 18:09.

Xtrend ET-9200, ET-8000, ET-10000, OpenPliPC on Ubuntu 12.04

Re: FTP Client (not as a plugin) #35 Erik Slagter

  • PLi® Core member
  • 46,951 posts

+541
Excellent

Posted 17 April 2015 - 20:30

Both scp and sftp use the same underlying transport (ssh). Maybe you are confused with ftps (like many people)?


* Wavefrontier T90 with 28E/23E/19E/13E via SCR switches 2 x 2 x 6 user bands
I don't read PM -> if you have something to ask or to report, do it in the forum so others can benefit. I don't take freelance jobs.
Ik lees geen PM -> als je iets te vragen of te melden hebt, doe het op het forum, zodat anderen er ook wat aan hebben.


Re: FTP Client (not as a plugin) #36 athoik

  • PLi® Core member
  • 8,458 posts

+327
Excellent

Posted 17 April 2015 - 20:33

SCP:  Can you give me an example to upload a file on a sftp server with username, password en rsa-key
And not the printout of the help file:
 
usage: scp [-1246BCpqrv] [-c cipher] [-F ssh_config] [-i identity_file]     
           [-l limit] [-P port] [-S program]                                
           [[user@]host1:]file1 [...] [[user@]host2:]file2     
 
I can get this myself.  I only cannot seems to figure out how to construct the command line.
 
Willy


Please have a look here: https://www.linux-ti...rsa-key-problem

Edited by athoik, 17 April 2015 - 20:34.

Wavefield T90: 0.8W - 1.9E - 4.8E - 13E - 16E - 19.2E - 23.5E - 26E - 33E - 39E - 42E - 45E on EMP Centauri DiseqC 16/1
Unamed: 13E Quattro - 9E Quattro on IKUSI MS-0916

Re: FTP Client (not as a plugin) #37 catastrofus

  • Senior Member
  • 3,440 posts

+43
Good

Posted 17 April 2015 - 21:10

You can use this example for your SFTP command: http://www.unix.com/...9eb01006ecb5dfd

You are a bit confusing in what you want. Giving a scp usage with username/password AND rsa key (?) while you just want to have an example of a sftp (batch) command...

 

SCP:  Can you give me an example to upload a file on a sftp server with username, password en rsa-key

And not the printout of the help file:

 

usage: scp [-1246BCpqrv] [-c cipher] [-F ssh_config] [-i identity_file]     
           [-l limit] [-P port] [-S program]                                
           [[user@]host1:]file1 [...] [[user@]host2:]file2     

 

I can get this myself.  I only cannot seems to figure out how to construct the command line.

 

Willy          


Edited by catastrofus, 17 April 2015 - 21:13.

2 x ultimo4k (dvb-c fbc + 1 dvb-s2, 8.3r/nb) + een et10k/osmini (geen tuners en dus fallbackclient, nb) met een Synology ds214+ (2 x 6 TB) op DSM 7.1.1 in ziggo oost (voormalig @Home) + A1/A2/A3/HB (TechniSat)
 


Re: FTP Client (not as a plugin) #38 doglover

  • Rytec EPG Team
  • 16,974 posts

+635
Excellent

Posted 18 April 2015 - 08:36

You are a bit confusing in what you want. Giving a scp usage with username/password AND rsa key (?) while you just want to have an example of a sftp (batch) command...

 

 

Yes this is what I want.

But there is here somebody which is trying to let it me do with scp.

 

Willy


~~Rytec Team~~
Maxytec Multibox SE OpenPli (used as mediaplayer)
Mutant HD2400 OpenPli
Vu+ Duo OpenPli (backup)

Synology NAS

Sat: 13E, 19.2E, 23.5E and 28.2E
*Pli/Rytec EPG POWERED*


Re: FTP Client (not as a plugin) #39 Erik Slagter

  • PLi® Core member
  • 46,951 posts

+541
Excellent

Posted 18 April 2015 - 08:45

But scp and sftp do the same, they both work on top of ssh. The difference is that scp works in "batch" mode and sftp is interactive, so if you CAN use ssh, I'd recommend to use scp, because you can use it from a script. You can also use wildcards (if properly escaped with quotes) and copy recursively.


* Wavefrontier T90 with 28E/23E/19E/13E via SCR switches 2 x 2 x 6 user bands
I don't read PM -> if you have something to ask or to report, do it in the forum so others can benefit. I don't take freelance jobs.
Ik lees geen PM -> als je iets te vragen of te melden hebt, doe het op het forum, zodat anderen er ook wat aan hebben.


Re: FTP Client (not as a plugin) #40 doglover

  • Rytec EPG Team
  • 16,974 posts

+635
Excellent

Posted 18 April 2015 - 09:11

OK.  I can log-in with SFTP manually.

Can even make a batch file to transfer files.

 

However it still requires me to type in the password manually.   I cannot find out how to pass the password in the batch file or from the command line.

 

BTW:  this is also the case for scp !!!

sftp -b /hdd/epg/king.txt user@IP

and in king.txt

cd /epg_data/
put /hdd/epg/rytecxmltvxxxxxx.gz

But this requires me to type in the password.  How can I pass this on automatically!!!!!

 

Willy

 

PS: it is obvious now I need an example, and not a referal to the webpage somewhere with some explanation.

I will try to find some examples with google.


~~Rytec Team~~
Maxytec Multibox SE OpenPli (used as mediaplayer)
Mutant HD2400 OpenPli
Vu+ Duo OpenPli (backup)

Synology NAS

Sat: 13E, 19.2E, 23.5E and 28.2E
*Pli/Rytec EPG POWERED*




Also tagged with one or more of these keywords: ET6000

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users