Additional dependencies

PyOpenSSL, to support FTPS:

$ pip install PyOpenSSL

pysendfile, if you’re on UNIX, in order to speedup uploads (from server to client):

$ pip install pysendfile

Table of Contents

Below is a set of example scripts showing some of the possible customizations that can be done with pyftpdlib. Some of them are included in demo directory of pyftpdlib source distribution.

A Base FTP server

The script below uses a basic configuration and it’s probably the best starting point to understand how things work. It uses the base DummyAuthorizer for adding a bunch of “virtual” users, sets a limit for incoming connections and a range of passive ports.

source code

import os
from pyftpdlib.authorizers import DummyAuthorizer
from pyftpdlib.handlers import FTPHandler
from pyftpdlib.servers import FTPServer
def main():


    # Instantiate a dummy authorizer for managing 'virtual' users


    authorizer = DummyAuthorizer()


    # Define a new user having full r/w permissions and a read-only


    # anonymous user


    authorizer.add_user('user', '12345', '.', perm='elradfmwMT')


    authorizer.add_anonymous(os.getcwd())


    # Instantiate FTP handler class


    handler = FTPHandler


    handler.authorizer = authorizer


    # Define a customized banner (string returned when client connects)


    handler.banner = "pyftpdlib based ftpd ready."


    # Specify a masquerade address and the range of ports to use for


    # passive connections.  Decomment in case you're behind a NAT.


    #handler.masquerade_address = '151.25.42.11'


    #handler.passive_ports = range(60000, 65535)


    # Instantiate FTP server class and listen on 0.0.0.0:2121


    address = ('', 2121)


    server = FTPServer(address, handler)


    # set a limit for connections


    server.max_cons = 256


    server.max_cons_per_ip = 5


    # start ftp server


    server.serve_forever()
if __name__ == '__main__':


    main()

Logging management

pyftpdlib uses the logging module to handle logging. If you don’t configure logging pyftpdlib will write logs to stderr. In order to configure logging you should do it before calling serve_forever(). Example logging to a file:

import logging
from pyftpdlib.handlers import FTPHandler
from pyftpdlib.servers import FTPServer
from pyftpdlib.authorizers import DummyAuthorizer
authorizer = DummyAuthorizer()
authorizer.add_user('user', '12345', '.', perm='elradfmwMT')
handler = FTPHandler
handler.authorizer = authorizer
logging.basicConfig(filename='/var/log/pyftpd.log', level=logging.INFO)
server = FTPServer(('', 2121), handler)
server.serve_forever()

DEBUG logging

You may want to enable DEBUG logging to observe commands and responses exchanged by client and server. DEBUG logging will also log internal errors which may occur on socket related calls such as send() and recv(). To enable DEBUG logging from code use:

logging.basicConfig(level=logging.DEBUG)

To enable DEBUG logging from command line use:

python -m pyftpdlib -D

DEBUG logs look like this:

[I 2017-11-07 12:03:44] >>> starting FTP server on 0.0.0.0:2121, pid=22991 <<<
[I 2017-11-07 12:03:44] concurrency model: async
[I 2017-11-07 12:03:44] masquerade (NAT) address: None
[I 2017-11-07 12:03:44] passive ports: None
[D 2017-11-07 12:03:44] poller: 'pyftpdlib.ioloop.Epoll'
[D 2017-11-07 12:03:44] authorizer: 'pyftpdlib.authorizers.DummyAuthorizer'
[D 2017-11-07 12:03:44] use sendfile(2): True
[D 2017-11-07 12:03:44] handler: 'pyftpdlib.handlers.FTPHandler'
[D 2017-11-07 12:03:44] max connections: 512
[D 2017-11-07 12:03:44] max connections per ip: unlimited
[D 2017-11-07 12:03:44] timeout: 300
[D 2017-11-07 12:03:44] banner: 'pyftpdlib 1.5.4 ready.'
[D 2017-11-07 12:03:44] max login attempts: 3
[I 2017-11-07 12:03:44] 127.0.0.1:37303-[] FTP session opened (connect)
[D 2017-11-07 12:03:44] 127.0.0.1:37303-[] -> 220 pyftpdlib 1.0.0 ready.
[D 2017-11-07 12:03:44] 127.0.0.1:37303-[] <- USER user
[D 2017-11-07 12:03:44] 127.0.0.1:37303-[] -> 331 Username ok, send password.
[D 2017-11-07 12:03:44] 127.0.0.1:37303-[user] <- PASS ******
[D 2017-11-07 12:03:44] 127.0.0.1:37303-[user] -> 230 Login successful.
[I 2017-11-07 12:03:44] 127.0.0.1:37303-[user] USER 'user' logged in.
[D 2017-11-07 12:03:44] 127.0.0.1:37303-[user] <- TYPE I
[D 2017-11-07 12:03:44] 127.0.0.1:37303-[user] -> 200 Type set to: Binary.
[D 2017-11-07 12:03:44] 127.0.0.1:37303-[user] <- PASV
[D 2017-11-07 12:03:44] 127.0.0.1:37303-[user] -> 227 Entering passive mode (127,0,0,1,233,208).
[D 2017-11-07 12:03:44] 127.0.0.1:37303-[user] <- RETR tmp-pyftpdlib
[D 2017-11-07 12:03:44] 127.0.0.1:37303-[user] -> 125 Data connection already open. Transfer starting.
[D 2017-11-07 12:03:44] 127.0.0.1:37303-[user] -> 226 Transfer complete.
[I 2017-11-07 12:03:44] 127.0.0.1:37303-[user] RETR /home/giampaolo/IMG29312.JPG completed=1 bytes=1205012 seconds=0.003
[D 2017-11-07 12:03:44] 127.0.0.1:37303-[user] <- QUIT
[D 2017-11-07 12:03:44] 127.0.0.1:37303-[user] -> 221 Goodbye.
[I 2017-11-07 12:03:44] 127.0.0.1:37303-[user] FTP session closed (disconnect).

Changing log line prefix

handler = FTPHandler
handler.log_prefix = 'XXX [%(username)s]@%(remote_ip)s'
server = FTPServer(('localhost', 2121), handler)
server.serve_forever()

Logs will now look like this:

[I 13-02-01 19:12:26] XXX []@127.0.0.1 FTP session opened (connect)
[I 13-02-01 19:12:26] XXX [user]@127.0.0.1 USER 'user' logged in.

Storing passwords as hash digests

Using FTP server library with the default DummyAuthorizer means that passwords will be stored in clear-text. An end-user ftpd using the default dummy authorizer would typically require a configuration file for authenticating users and their passwords but storing clear-text passwords is of course undesirable. The most common way to do things in such case would be first creating new users and then storing their usernames + passwords as hash digests into a file or wherever you find it convenient. The example below shows how to storage passwords as one-way hashes by using md5 algorithm.

source code

import os
import sys
from hashlib import md5
from pyftpdlib.handlers import FTPHandler
from pyftpdlib.servers import FTPServer
from pyftpdlib.authorizers import DummyAuthorizer, AuthenticationFailed
class DummyMD5Authorizer(DummyAuthorizer):


    def validate_authentication(self, username, password, handler):


        if sys.version_info >= (3, 0):


            password = md5(password.encode('latin1'))


        hash = md5(password).hexdigest()


        try:


            if self.user_table[username]['pwd'] != hash:


                raise KeyError


        except KeyError:


            raise AuthenticationFailed
def main():


    # get a hash digest from a clear-text password


    hash = md5('12345').hexdigest()


    authorizer = DummyMD5Authorizer()


    authorizer.add_user('user', hash, os.getcwd(), perm='elradfmwMT')


    authorizer.add_anonymous(os.getcwd())


    handler = FTPHandler


    handler.authorizer = authorizer


    server = FTPServer(('', 2121), handler)


    server.serve_forever()
if __name__ == "__main__":


    main()

Unix FTP Server

If you’re running a Unix system you may want to configure your ftpd to include support for “real” users existing on the system and navigate the real filesystem. The example below uses UnixAuthorizer and UnixFilesystem classes to do so.

from pyftpdlib.handlers import FTPHandler
from pyftpdlib.servers import FTPServer
from pyftpdlib.authorizers import UnixAuthorizer
from pyftpdlib.filesystems import UnixFilesystem
def main():


    authorizer = UnixAuthorizer(rejected_users=["root"], require_valid_shell=True)


    handler = FTPHandler


    handler.authorizer = authorizer


    handler.abstracted_fs = UnixFilesystem


    server = FTPServer(('', 21), handler)


    server.serve_forever()
if __name__ == "__main__":


    main()

Windows FTP Server

The following code shows how to implement a basic authorizer for a Windows NT workstation to authenticate against existing Windows user accounts. This code requires Mark Hammond’s pywin32 extension to be installed.

source code

from pyftpdlib.handlers import FTPHandler
from pyftpdlib.servers import FTPServer
from pyftpdlib.authorizers import WindowsAuthorizer
def main():


    authorizer = WindowsAuthorizer()


    # Use Guest user with empty password to handle anonymous sessions.


    # Guest user must be enabled first, empty password set and profile


    # directory specified.


    #authorizer = WindowsAuthorizer(anonymous_user="Guest", anonymous_password="")


    handler = FTPHandler


    handler.authorizer = authorizer


    server = FTPServer(('', 2121), handler)


    server.serve_forever()
if __name__ == "__main__":


    main()

Changing the concurrency model

By nature pyftpdlib is asynchronous. That means it uses a single process/thread to handle multiple client connections and file transfers. This is why it is so fast, lightweight and scalable (see benchmarks). The async model has one big drawback though: the code cannot contain instructions which blocks for a long period of time, otherwise the whole FTP server will hang. As such the user should avoid calls such as time.sleep(3), heavy db queries, etc. Moreover, there are cases where the async model is not appropriate, and that is when you’re dealing with a particularly slow filesystem (say a network filesystem such as samba). If the filesystem is slow (say, a open(file, 'r').read(8192) takes 2 secs to complete) then you are stuck. Starting from version 1.0.0 pyftpdlib can change the concurrency model by using multiple processes or threads instead. In technical (internal) terms that means that every time a client connects a separate thread/process is spawned and internally it will run its own IO loop. In practical terms this means that you can block as long as you want. Changing the concurrency module is easy: you just need to import a substitute for FTPServer. class:

Multiple threads

from pyftpdlib.handlers import FTPHandler
from pyftpdlib.servers import ThreadedFTPServer  # <-
from pyftpdlib.authorizers import DummyAuthorizer
def main():


    authorizer = DummyAuthorizer()


    authorizer.add_user('user', '12345', '.')


    handler = FTPHandler


    handler.authorizer = authorizer


    server = ThreadedFTPServer(('', 2121), handler)


    server.serve_forever()
if __name__ == "__main__":


    main()

Multiple processes

from pyftpdlib.handlers import FTPHandler
from pyftpdlib.servers import MultiprocessFTPServer  # <-
from pyftpdlib.authorizers import DummyAuthorizer
def main():


    authorizer = DummyAuthorizer()


    authorizer.add_user('user', '12345', '.')


    handler = FTPHandler


    handler.authorizer = authorizer


    server = MultiprocessFTPServer(('', 2121), handler)


    server.serve_forever()
if __name__ == "__main__":


    main()

Pre-fork

There also exists a third option (UNIX only): the pre-fork model. Pre-fork means that a certain number of worker processes are

``

spawn()``ed before starting the server. Each worker process will keep using a 1-thread, async concurrency model, handling multiple concurrent connections, but the workload is split. This way the delay introduced by a blocking function call is amortized and divided by the number of workers, and thus also the disk I/O latency is minimized. Every time a new connection comes in, the parent process will automatically delegate the connection to one of the subprocesses, so from the app standpoint this is completely transparent. As a general rule, it is always a good idea to use this model in production. The optimal value depends on many factors including (but not limited to) the number of CPU cores, the number of hard disk drives that store data, and load pattern. When one is in doubt, setting it to the number of available CPU cores would be a good start.

from pyftpdlib.handlers import FTPHandler
from pyftpdlib.servers import FTPServer
from pyftpdlib.authorizers import DummyAuthorizer
def main():


    authorizer = DummyAuthorizer()


    authorizer.add_user('user', '12345', '.')


    handler = FTPHandler


    handler.authorizer = authorizer


    server = FTPServer(('', 2121), handler)


    server.serve_forever(worker_processes=4)  # <-
if __name__ == "__main__":


    main()

Throttle bandwidth

An important feature for an ftpd is limiting the speed for downloads and uploads affecting the data channel. ThrottledDTPHandler.banner can be used to set such limits. The basic idea behind ThrottledDTPHandler is to wrap sending and receiving in a data counter and temporary “sleep” the data channel so that you burst to no more than x Kb/sec average. When it realizes that more than x Kb in a second are being transmitted it temporary blocks the transfer for a certain number of seconds.

import os
from pyftpdlib.handlers import FTPHandler, ThrottledDTPHandler
from pyftpdlib.servers import FTPServer
from pyftpdlib.authorizers import DummyAuthorizer
def main():


    authorizer = DummyAuthorizer()


    authorizer.add_user('user', '12345', os.getcwd(), perm='elradfmwMT')


    authorizer.add_anonymous(os.getcwd())


    dtp_handler = ThrottledDTPHandler


    dtp_handler.read_limit = 30720  # 30 Kb/sec (30 * 1024)


    dtp_handler.write_limit = 30720  # 30 Kb/sec (30 * 1024)


    ftp_handler = FTPHandler


    ftp_handler.authorizer = authorizer


    # have the ftp handler use the alternative dtp handler class


    ftp_handler.dtp_handler = dtp_handler


    server = FTPServer(('', 2121), ftp_handler)


    server.serve_forever()
if __name__ == '__main__':


    main()

FTPS (FTP over TLS/SSL) server

Starting from version 0.6.0 pyftpdlib finally includes full FTPS support implementing both TLS and SSL protocols and AUTH, PBSZ and PROT commands as defined in RFC-4217. This has been implemented by using PyOpenSSL module, which is required in order to run the code below. TLS_FTPHandler class requires at least a certfile to be specified and optionally a keyfile. Apache FAQs provide instructions on how to generate them. If you don’t care about having your personal self-signed certificates you can use the one in the demo directory which include both and is available here.

source code

"""
An RFC-4217 asynchronous FTPS server supporting both SSL and TLS.
Requires PyOpenSSL module (http://pypi.python.org/pypi/pyOpenSSL).
"""
from pyftpdlib.servers import FTPServer
from pyftpdlib.authorizers import DummyAuthorizer
from pyftpdlib.handlers import TLS_FTPHandler
def main():


    authorizer = DummyAuthorizer()


    authorizer.add_user('user', '12345', '.', perm='elradfmwMT')


    authorizer.add_anonymous('.')


    handler = TLS_FTPHandler


    handler.certfile = 'keycert.pem'


    handler.authorizer = authorizer


    # requires SSL for both control and data channel


    #handler.tls_control_required = True


    #handler.tls_data_required = True


    server = FTPServer(('', 21), handler)


    server.serve_forever()
if __name__ == '__main__':


    main()

Event callbacks

A small example which shows how to use callback methods via FTPHandler subclassing:

from pyftpdlib.handlers import FTPHandler
from pyftpdlib.servers import FTPServer
from pyftpdlib.authorizers import DummyAuthorizer
class MyHandler(FTPHandler):


    def on_connect(self):


        print "%s:%s connected" % (self.remote_ip, self.remote_port)


    def on_disconnect(self):


        # do something when client disconnects


        pass


    def on_login(self, username):


        # do something when user login


        pass


    def on_logout(self, username):


        # do something when user logs out


        pass


    def on_file_sent(self, file):


        # do something when a file has been sent


        pass


    def on_file_received(self, file):


        # do something when a file has been received


        pass


    def on_incomplete_file_sent(self, file):


        # do something when a file is partially sent


        pass


    def on_incomplete_file_received(self, file):


        # remove partially uploaded files


        import os


        os.remove(file)
def main():


    authorizer = DummyAuthorizer()


    authorizer.add_user('user', '12345', homedir='.', perm='elradfmwMT')


    authorizer.add_anonymous(homedir='.')


    handler = MyHandler


    handler.authorizer = authorizer


    server = FTPServer(('', 2121), handler)


    server.serve_forever()
if __name__ == "__main__":


    main()

Command line usage

Starting from version 0.6.0 pyftpdlib can be run as a simple stand-alone server via Python’s -m option, which is particularly useful when you want to quickly share a directory. Some examples. Anonymous FTPd sharing current directory:

$ python -m pyftpdlib
[I 13-04-09 17:55:18] >>> starting FTP server on 0.0.0.0:2121, pid=6412 <<<
[I 13-04-09 17:55:18] poller: <class 'pyftpdlib.ioloop.Epoll'>
[I 13-04-09 17:55:18] masquerade (NAT) address: None
[I 13-04-09 17:55:18] passive ports: None
[I 13-04-09 17:55:18] use sendfile(2): True

Anonymous FTPd with write permission:

$ python -m pyftpdlib -w

Set a different address/port and home directory:

$ python -m pyftpdlib -i localhost -p 8021 -d /home/someone

See python -m pyftpdlib -h for a complete list of options.

Table of Contents

pyftpdlib implements the server side of the FTP protocol as defined in RFC-959. This document is intended to serve as a simple API reference of most important classes and functions. After reading this you will probably want to read the tutorial including customization through the use of some example scripts.

Modules and classes hierarchy

pyftpdlib.authorizers
pyftpdlib.authorizers.AuthenticationFailed
pyftpdlib.authorizers.DummyAuthorizer
pyftpdlib.authorizers.UnixAuthorizer
pyftpdlib.authorizers.WindowsAuthorizer
pyftpdlib.handlers
pyftpdlib.handlers.FTPHandler
pyftpdlib.handlers.TLS_FTPHandler
pyftpdlib.handlers.DTPHandler
pyftpdlib.handlers.TLS_DTPHandler
pyftpdlib.handlers.ThrottledDTPHandler
pyftpdlib.filesystems
pyftpdlib.filesystems.FilesystemError
pyftpdlib.filesystems.AbstractedFS
pyftpdlib.filesystems.UnixFilesystem
pyftpdlib.servers
pyftpdlib.servers.FTPServer
pyftpdlib.servers.ThreadedFTPServer
pyftpdlib.servers.MultiprocessFTPServer
pyftpdlib.ioloop
pyftpdlib.ioloop.IOLoop
pyftpdlib.ioloop.Connector
pyftpdlib.ioloop.Acceptor
pyftpdlib.ioloop.AsyncChat

Users

Control connection

Data connection

Server (acceptor)

Filesystem

Extended classes

Extended handlers

Extended authorizers

Extended filesystems

Extended servers

Table of Contents

Introduction

What is pyftpdlib?

pyftpdlib is a high-level library to easily write asynchronous portable FTP servers with Python.

What is Python?

Python is an interpreted, interactive, object-oriented, easy-to-learn programming language. It is often compared to Tcl, Perl, Scheme or Java.

I’m not a python programmer. Can I use it anyway?

Yes. pyftpdlib is a fully working FTP server implementation that can be run “as is”. For example you could run an anonymous ftp server from cmd-line by running:

$ sudo python -m pyftpdlib
[I 13-02-20 14:16:36] >>> starting FTP server on 0.0.0.0:8021 <<<
[I 13-02-20 14:16:36] poller: <class 'pyftpdlib.ioloop.Epoll'>
[I 13-02-20 14:16:36] masquerade (NAT) address: None
[I 13-02-20 14:16:36] passive ports: None
[I 13-02-20 14:16:36] use sendfile(2): True

This is useful in case you want a quick and dirty way to share a directory without, say, installing and configuring samba. Starting from version 0.6.0 options can be passed to the command line (see python -m pyftpdlib --help to see all available options). Examples:

Anonymous FTP server with write access:

$ sudo python -m pyftpdlib -w
~pyftpdlib-1.3.1-py2.7.egg/pyftpdlib/authorizers.py:265: RuntimeWarning: write permissions assigned to anonymous user.
[I 13-02-20 14:16:36] >>> starting FTP server on 0.0.0.0:8021 <<<
[I 13-02-20 14:16:36] poller: <class 'pyftpdlib.ioloop.Epoll'>
[I 13-02-20 14:16:36] masquerade (NAT) address: None
[I 13-02-20 14:16:36] passive ports: None
[I 13-02-20 14:16:36] use sendfile(2): True

Listen on a different ip/port:

$ python -m pyftpdlib -i 127.0.0.1 -p 8021
[I 13-02-20 14:16:36] >>> starting FTP server on 0.0.0.0:8021 <<<
[I 13-02-20 14:16:36] poller: <class 'pyftpdlib.ioloop.Epoll'>
[I 13-02-20 14:16:36] masquerade (NAT) address: None
[I 13-02-20 14:16:36] passive ports: None
[I 13-02-20 14:16:36] use sendfile(2): True

Customizing ftpd for basic tasks like adding users or deciding where log file should be placed is mostly simply editing variables. This is basically like learning how to edit a common unix ftpd.conf file and doesn’t really require Python knowledge. Customizing ftpd more deeply requires a python script which imports pyftpdlib to be written separately. An example about how this could be done are the scripts contained in the demo directory.

Getting help

There’s a mailing list available at: http://groups.google.com/group/pyftpdlib/topics

Installing and compatibility

How do I install pyftpdlib?

If you are not new to Python you probably don’t need that, otherwise follow the install instructions.

Which Python versions are compatible?

From 2.6 to 3.X.

On which platforms can pyftpdlib be used?

pyftpdlib should work on any platform where select(), poll(), epoll() or kqueue() system calls are available and on any Python implementation which refers to cPython 2.6 or superior. The development team has mainly tested it under various Linux, Windows, OSX and FreeBSD systems. For FreeBSD is also available a pre-compiled package maintained by Li-Wen Hsu (lwhsu@freebsd.org). Other Python implementation like PythonCE are known to work with pyftpdlib and every new version is usually tested against it. pyftpdlib currently does not work on Jython since the latest Jython release refers to CPython 2.2.x serie. The best way to know whether pyftpdlib works on your platform is installing it and running its test suite.

Usage

How can I run long-running tasks without blocking the server?

pyftpdlib is an asynchronous FTP server. That means that if you need to run a time consuming task you have to use a separate Python process or thread for the actual processing work otherwise the entire asynchronous loop will be blocked.

Let’s suppose you want to implement a long-running task every time the server receives a file. The code snippet below shows the correct way to do it by using a thread.

With self.del_channel() we temporarily “sleep” the connection handler which will be removed from the async IO poller loop and won’t be able to send or receive any more data. It won’t be closed (disconnected) as long as we don’t invoke self.add_channel(). This is fundamental when working with threads to avoid race conditions, dead locks etc.

class MyHandler(FTPHandler):


    def on_file_received(self, file):


        def blocking_task():


            time.sleep(5)


            self.add_channel()


        self.del_channel()


        threading.Thread(target=blocking_task).start()

Another possibility is to change the default concurrency model.

Why do I get socket.error “Permission denied” error on ftpd starting?

Probably because you’re on a Unix system and you’re trying to start ftpd as an unprivileged user. FTP servers bind on port 21 by default and only super-user account (e.g. root) can bind sockets on such ports. If you want to bind ftpd as non-privileged user you should set a port higher than 1024.

How can I prevent the server version from being displayed?

Just modify FTPHandler.banner.

Can control upload/download ratios?

Yes. Starting from version 0.5.2 pyftpdlib provides a new class called ThrottledDTPHandler. You can set speed limits by modifying read_limit and write_limit class attributes as it is shown in throttled_ftpd.py demo script.

Are there ways to limit connections?

FTPServer. class comes with two overridable attributes defaulting to zero (no limit): max_cons, which sets a limit for maximum simultaneous connection to handle by ftpd and max_cons_per_ip which set a limit for connections from the same IP address. Overriding these variables is always recommended to avoid DoS attacks.

I’m behind a NAT / gateway

When behind a NAT a ftp server needs to replace the IP local address displayed in PASV replies and instead use the public address of the NAT to allow client to connect. By overriding masquerade_address attribute of FTPHandler class you will force pyftpdlib to do such replacement. However, one problem still exists. The passive FTP connections will use ports from 1024 and up, which means that you must forward all ports 1024-65535 from the NAT to the FTP server! And you have to allow many (possibly) dangerous ports in your firewalling rules! To resolve this, simply override passive_ports attribute of FTPHandler class to control what ports pyftpdlib will use for its passive data transfers. Value expected by passive_ports attribute is a list of integers (e.g. range(60000, 65535)) indicating which ports will be used for initializing the passive data channel. In case you run a FTP server with multiple private IP addresses behind a NAT firewall with multiple public IP addresses you can use passive_ports option which allows you to define multiple 1 to 1 mappings (New in 0.6.0).

What is FXP?

FXP is part of the name of a popular Windows FTP client: http://www.flashfxp.com. This client has made the name “FXP” commonly used as a synonym for site-to-site FTP transfers, for transferring a file between two remote FTP servers without the transfer going through the client’s host. Sometimes “FXP” is referred to as a protocol; in fact, it is not. The site-to-site transfer capability was deliberately designed into RFC-959. More info can be found here: http://www.proftpd.org/docs/howto/FXP.html.

Does pyftpdlib support FXP?

Yes. It is disabled by default for security reasons (see RFC-2257 and FTP bounce attack description) but in case you want to enable it just set to True the permit_foreign_addresses attribute of FTPHandler class.

Why timestamps shown by MDTM and ls commands (LIST, MLSD, MLST) are wrong?

If by “wrong” you mean “different from the timestamp of that file on my client machine”, then that is the expected behavior. Starting from version 0.6.0 pyftpdlib uses GMT times as recommended in RFC-3659. In case you want such commands to report local times instead just set the use_gmt_times attribute to False. For further information you might want to take a look at this

Implementation

sendfile()

Starting from version 0.7.0 if pysendfile module is installed sendfile(2) system call be used when uploading files (from server to client) via RETR command. Using sendfile(2) usually results in transfer rates from 2x to 3x faster and less CPU usage. Note: use of sendfile() might introduce some unexpected issues with “non regular filesystems” such as NFS, SMBFS/Samba, CIFS and network mounts in general, see: http://www.proftpd.org/docs/howto/Sendfile.html. If you bump into one this problems the fix consists in disabling sendfile() usage via FTPHandler.use_sendfile option:

from pyftpdlib.handlers import FTPHandler
handler = FTPHandler
handler.use_senfile = False
...

Globbing / STAT command implementation

Globbing is a common Unix shell mechanism for expanding wildcard patterns to match multiple filenames. When an argument is provided to the STAT command, ftpd should return directory listing over the command channel. RFC-959 does not explicitly mention globbing; this means that FTP servers are not required to support globbing in order to be compliant. However, many FTP servers do support globbing as a measure of convenience for FTP clients and users. In order to search for and match the given globbing expression, the code has to search (possibly) many directories, examine each contained filename, and build a list of matching files in memory. Since this operation can be quite intensive, both CPU- and memory-wise, pyftpdlib does not support globbing.

ASCII transfers / SIZE command implementation

Properly handling the SIZE command when TYPE ASCII is used would require to scan the entire file to perform the ASCII translation logic (file.read().replace(os.linesep, ‘rn’)) and then calculating the len of such data which may be different than the actual size of the file on the server. Considering that calculating such result could be very resource-intensive it could be easy for a malicious client to try a DoS attack, thus pyftpdlib rejects SIZE when the current TYPE is ASCII. However, clients in general should not be resuming downloads in ASCII mode. Resuming downloads in binary mode is the recommended way as specified in RFC-3659.

IPv6 support

Starting from version 0.4.0 pyftpdlib supports IPv6 (RFC-2428). If you use IPv6 and want your FTP daemon to do so just pass a valid IPv6 address to the FTPServer class constructor. Example:

>>> from pyftpdlib.servers import FTPServer
>>> address = ("::1", 21)  # listen on localhost, port 21
>>> ftpd = FTPServer(address, FTPHandler)
>>> ftpd.serve_forever()
Serving FTP on ::1:21

If your OS (for example: all recent UNIX systems) have an hybrid dual-stack IPv6/IPv4 implementation the code above will listen on both IPv4 and IPv6 by using a single IPv6 socket (New in 0.6.0).

How do I install IPv6 support on my system?

If you want to install IPv6 support on Linux run “modprobe ipv6”, then “ifconfig”. This should display the loopback adapter, with the address “::1”. You should then be able to listen the server on that address, and connect to it. On Windows (XP SP2 and higher) run “netsh int ipv6 install”. Again, you should be able to use IPv6 loopback afterwards.

Can pyftpdlib be integrated with “real” users existing on the system?

Yes. Starting from version 0.6.0 pyftpdlib provides the new UnixAuthorizer and WindowsAuthorizer classes. By using them pyftpdlib can look into the system account database to authenticate users. They also assume the id of real users every time the FTP server is going to access the filesystem (e.g. for creating or renaming a file) the authorizer will temporarily impersonate the currently logged on user, execute the filesystem call and then switch back to the user who originally started the server. Example UNIX and Windows FTP servers contained in the demo directory shows how to use UnixAuthorizer and WindowsAuthorizer classes.

Does pyftpdlib support FTP over TLS/SSL (FTPS)?

Yes, starting from version 0.6.0, see: Does pyftpdlib support FTP over TLS/SSL (FTPS)?

What about SITE commands?

The only supported SITE command is SITE CHMOD (change file mode). The user willing to add support for other specific SITE commands has to define a new ftp_SITE_CMD method in the FTPHandler subclass and add a new entry in proto_cmds dictionary. Example:

from pyftpdlib.handlers import FTPHandler
proto_cmds = FTPHandler.proto_cmds.copy()
proto_cmds.update(


    {'SITE RMTREE': dict(perm='R', auth=True, arg=True,


      help='Syntax: SITE <SP> RMTREE <SP> path (remove directory tree).')}
)
class CustomizedFTPHandler(FTPHandler):


    proto_cmds = proto_cmds
def ftp_SITE_RMTREE(self, line):


    """Recursively remove a directory tree."""


    # implementation here


    # ...

pyftpdlib 0.7.0 vs. pyftpdlib 1.0.0

pyftpdlib vs. proftpd 1.3.4

pyftpdlib vs. vsftpd 2.3.5

pyftpdlib vs. Twisted 12.3

By using sendfile() (Twisted does not support sendfile()):

By using plain send():

Memory usage

Values on UNIX are calculated as (rss - shared).

Interpreting the results

pyftpdlib and proftpd / vsftpd look pretty much equally fast. The huge difference is noticeable in scalability though, because of the concurrency model adopted. Both proftpd and vsftpd spawn a new process for every connected client, where pyftpdlib doesn’t (see the C10k problem). The outcome is well noticeable on connect/login benchmarks and memory benchmarks.

The huge differences between 0.7.0 and 1.0.0 versions of pyftpdlib are due to fix of issue 203. On Linux we now use epoll() which scales considerably better than select(). The fact that we’re downloading a file with 300 idle clients doesn’t make any difference for epoll(). We might as well had 5000 idle clients and the result would have been the same. On Windows, where we still use select(), 1.0.0 still wins hands down as the asyncore loop was reimplemented from scratch in order to support fd un/registration and modification (see issue 203). All the benchmarks were conducted on a Linux Ubuntu 12.04 Intel core duo - 3.1 Ghz box.

Setup

The following setup was used before running every benchmark:

proftpd

# /etc/proftpd/proftpd.conf
MaxInstances        2000

…followed by:

$ sudo service proftpd restart

vsftpd

# /etc/vsftpd.conf
local_enable=YES
write_enable=YES
max_clients=2000
max_per_ip=2000

…followed by:

$ sudo service vsftpd restart

twisted FTP server

from twisted.protocols.ftp import FTPFactory, FTPRealm
from twisted.cred.portal import Portal
from twisted.cred.checkers import AllowAnonymousAccess, FilePasswordDB
from twisted.internet import reactor
import resource
soft, hard = resource.getrlimit(resource.RLIMIT_NOFILE)
resource.setrlimit(resource.RLIMIT_NOFILE, (hard, hard))
open('pass.dat', 'w').write('user:some-passwd')
p = Portal(FTPRealm('./'),
[AllowAnonymousAccess(), FilePasswordDB("pass.dat")])
f = FTPFactory(p)
reactor.listenTCP(21, f)
reactor.run()

…followed by:

$ sudo python twist_ftpd.py

pyftpdlib

The following patch was applied first:

Index: pyftpdlib/servers.py
===================================================================
--- pyftpdlib/servers.py    (revisione 1154)
+++ pyftpdlib/servers.py    (copia locale)
@@ -494,3 +494,10 @@
def _map_len(self):
return len(multiprocessing.active_children())
+
+import resource
+soft, hard = resource.getrlimit(resource.RLIMIT_NOFILE)
+resource.setrlimit(resource.RLIMIT_NOFILE, (hard, hard))
+FTPServer.max_cons = 0

…followed by:

$ sudo python demo/unix_daemon.py

The benchmark script was run as:

python scripts/ftpbench -u USERNAME -p PASSWORD -b all -n 300

…and for the memory test:

python scripts/ftpbench -u USERNAME -p PASSWORD -b all -n 300 -k FTP_SERVER_PID

Table of Contents

Introduction

This page lists current standard Internet RFCs that define the FTP protocol.

pyftpdlib conforms to the FTP protocol standard as defined in RFC-959 and RFC-1123 implementing all the fundamental commands and features described in them. It also implements some more recent features such as OPTS and FEAT commands (RFC-2398), EPRT and EPSV commands covering the IPv6 support (RFC-2428) and MDTM, MLSD, MLST and SIZE commands defined in RFC-3659.

Future plans for pyftpdlib include the gradual implementation of other standards track RFCs.

Some of the features like ACCT or SMNT commands will never be implemented deliberately. Other features described in more recent RFCs like the TLS/SSL support for securing FTP (RFC-4217) are now implemented as a demo script, waiting to reach the proper level of stability to be then included in the standard code base.

RFC-959 - File Transfer Protocol

The base specification of the current File Transfer Protocol.

RFC-1123 - Requirements for Internet Hosts

Extends and clarifies some aspects of RFC-959. Introduces new response codes 554 and 555.

RFC-2228 - FTP Security Extensions

Specifies several security extensions to the base FTP protocol defined in RFC-959. New commands: AUTH, ADAT, PROT, PBSZ, CCC, MIC, CONF, and ENC. New response codes: 232, 234, 235, 334, 335, 336, 431, 533, 534, 535, 536, 537, 631, 632, and 633.

RFC-2389 - Feature negotiation mechanism for the File Transfer Protocol

Introduces the new FEAT and OPTS commands.

RFC-2428 - FTP Extensions for IPv6 and NATs

Introduces the new commands EPRT and EPSV extending FTP to enable its use over various network protocols, and the new response codes 522 and 229.

RFC-2577 - FTP Security Considerations

Provides several configuration and implementation suggestions to mitigate some security concerns, including limiting failed password attempts and third-party “proxy FTP” transfers, which can be used in “bounce attacks”.

RFC-2640 - Internationalization of the File Transfer Protocol

Extends the FTP protocol to support multiple character sets, in addition to the original 7-bit ASCII. Introduces the new LANG command.

RFC-3659 - Extensions to FTP

Four new commands are added: “SIZE”, “MDTM”, “MLST”, and “MLSD”. The existing command “REST” is modified.

RFC-4217 - Securing FTP with TLS

Provides a description on how to implement TLS as a security mechanism to secure FTP clients and/or servers.

Unofficial commands

These are commands not officialy included in any RFC but many FTP servers implement them.

Table of Contents

Here comes a list of softwares and systems using pyftpdlib. In case you want to add your software to such list add a comment below. Please help us in keeping such list updated.

Packages

Following lists the packages of pyftpdlib from various platforms.

Debian

[image]

A .deb packaged version of pyftpdlib is available.

Fedora

[image]

A RPM packaged version is available.

FreeBSD

[image]

A freshport is available.

GNU Darwin

[image]

GNU Darwin is a Unix distribution which focuses on the porting of free software to Darwin and Mac OS X. pyftpdlib has been recently included in the official repositories to make users can easily install and use it on GNU Darwin systems.

Softwares

Following lists the softwares adopting pyftpdlib.

Google Chrome

[image]

Google Chrome is the new free and open source web browser developed by Google. Google Chromium, the open source project behind Google Chrome, included pyftpdlib in the code base to develop Google Chrome’s FTP client unit tests.

Smartfile

[image]

Smartfile is a market leader in FTP and online file storage that has a robust and easy-to-use web interface. We utilize pyftpdlib as the underpinnings of our FTP service. Pyftpdlib gives us the flexibility we require to integrate FTP with the rest of our application.

Bazaar

[image]

Bazaar is a distributed version control system similar to Subversion which supports different protocols among which FTP. As for Google Chrome, Bazaar recently adopted pyftpdlib as base FTP server to implement internal FTP unit tests.

Python for OpenVMS

[image]

OpenVMS is an operating system that runs on the VAX and Alpha families of computers, now owned by Hewlett-Packard. vmspython is a porting of the original cPython interpreter that runs on OpenVMS platforms. pyftpdlib recently became a standard library module installed by default on every new vmspython installation.

http://www.vmspython.org/DownloadAndInstallationPython

OpenERP

[image]

OpenERP is an Open Source enterprise management software. It covers and integrates most enterprise needs and processes: accounting, hr, sales, crm, purchase, stock, production, services management, project management, marketing campaign, management by affairs. OpenERP recently included pyftpdlib as plug in to serve documents via FTP.

Plumi

[image]

Plumi is a video sharing Content Management System based on Plone that enables you to create your own sophisticated video sharing site. pyftpdlib has been included in Plumi to allow resumable large video file uploads into Zope.

put.io FTP connector

[image]

A proof of concept FTP server that proxies FTP clients requests to putio via HTTP, or in other words an FTP interface to put.io Put.io is a storage service that fetches media files remotely and lets you stream them immediately. More info can be found here. See https://github.com/ybrs/putio-ftp-connector blog entry

Rackspace Cloud’s FTP

[image]

ftp-cloudfs is a ftp server acting as a proxy to Rackspace Cloud Files. It allows you to connect via any FTP client to do upload/download or create containers.

Far Manager

[image]

Far Manager is a program for managing files and archives in Windows operating systems. Far Manager recently included pyftpdlib as a plug-in for making the current directory accessible through FTP. Convenient for exchanging files with virtual machines.

Google Pages FTPd

[image]

gpftpd is a pyftpdlib based FTP server you can connect to using your Google e-mail account. It redirects you to all files hosted on your Google Pages account giving you access to download them and upload new ones.

Peerscape

[image]

Peerscape is an experimental peer-to-peer social network implemented as an extension to the Firefox web browser. It implements a kind of serverless read-write web supporting third-party AJAX application development. Under the hood, your computer stores copies of your data, the data of your friends and the groups you have joined, and some data about, e.g., friends of friends. It also caches copies of other data that you navigate to. Computers that store the same data establish connections among themselves to keep it in sync.

feitp-server

An extra layer on top of pyftpdlib introducing multi processing capabilities and overall higher performances.

Symbian Python FTP server

[image]

An FTP server for Symbian OS: http://code.google.com/p/sypftp/

ftp-cloudfs

An FTP server acting as a proxy to Rackspace Cloud Files or to OpenStack Swift. It allow you to connect via any FTP client to do upload/download or create containers: https://github.com/chmouel/ftp-cloudfs

Sierramobilepos

The goal of this project is to extend Openbravo POS to use Windows Mobile Professional or Standard devices. It will import the data from Ob POS (originally in Postgres, later MySql). This data will reside in a database using sqlite3. Later a program will allow to sync by FTP or using a USB cable connected to the WinMob device. link

Faetus

Faetus is a FTP server that translates FTP commands into Amazon S3 API calls providing an FTP interface on top of Amazon S3 storage.

Pyfilesystem

Pyfilesystem is a Python module that provides a common interface to many types of filesystem, and provides some powerful features such as exposing filesystems over an internet connection, or to the native filesystem. It uses pyftpdlib as a backend for testing its FTP component.

Manent

Manent is an algorithmically strong backup and archival program which can offer remote backup via a pyftpdlib-based S/FTP server.

Aksy

Aksy is a Python module to control S5000/S6000, Z4/Z8 and MPC4000 Akai sampler models with System Exclusive over USB. Aksy introduced the possibility to mount samplers as web folders and manage files on the sampler via FTP.

Imgserve

Imgserve is a python image processing server designed to provide image processing service. It can utilize modern multicore CPU to achieve higher throughput and possibly better performance. It uses pyftpdlib to permit image downloading/uploading through FTP/FTPS.

Shareme

Ever needed to share a directory between two computers? Usually this is done using NFS, FTP or Samba, which could be a pain to setup when you just want to move some files around. Shareme is a small FTP server that, without configuration files or manuals to learn, will publish your directory, and users can download from it and upload files and directory. Just open a shell and run shareme -d ~/incoming/ …and that’s it!

Zenftp

A simple service that bridges an FTP client with zenfolio via SOAP. Start zenftp.py, providing the name of the target photoset on Zenfolio, and then connect to localhost with your FTP client. link

ftpmaster

A very simple FTP-based content management system (CMS) including an LDAP authorizer. link

ShareFTP

A program functionally equivalent to Shareme project. link

EasyFTPd

An end-user UNIX FTP server with focus on simplicity. It basically provides a configuration file interface over pyftpdlib to easily set up an FTP daemon. link.

Eframe

Eframe offers Python support for the BT EFrame 1000 digital photo frame.

Fastersync

A tool to synchronize data between desktop PCs, laptops, USB drives, remote FTP/SFTP servers, and different online data storages. link

bftpd

A small easy to configure FTP server. link

Web sites using pyftpdlib

www.bitsontherun.com

[image]

http://www.bitsontherun.com

www.adcast.tv

[image]

http://www.adcast.tv http://www.adcast.tv

www.netplay.it

[image]

http://netplay.it/