git.seregonwar.com

README.md

zftpd / README.md

1	<div align="center">
2
3	<img src="assets/zftpd-logo.png" alt="zftpd" width="75%" />
4
5	<br/><br/>
6
7	A zero-copy FTP daemon built for speed, correctness, and portability.
8	Runs anywhere POSIX runs. Saturates Gigabit. Ships a console payload too.
9
10	<br/>
11
12	[![C11](https://img.shields.io/badge/C-11-informational?style=flat-square&logo=c&logoColor=white)](https://en.cppreference.com/w/c/11)
13	[![MIT](https://img.shields.io/badge/license-MIT-informational?style=flat-square)](LICENSE)
14	[![Downloads](https://img.shields.io/github/downloads/seregonwar/zftpd/total?style=flat-square&color=brightgreen)](https://github.com/seregonwar/zftpd/releases)
15	[![Platform](https://img.shields.io/badge/platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20PS4%20%7C%20PS5-blueviolet?style=flat-square)](#build)
16
17	<br/>
18
19	[Overview](#overview) · [Performance](#-performance) · [Features](#-features) · [Best Setup](#-best-setup) · [Build](#-build) · [Running](#-running) · [Configuration](#-configuration) · [ZHTTP](#-zhttp)
20
21	<br/>
22
23	</div>
24
25	---
26
27	## Overview
28
29	`zftpd` is a high-performance FTP server written in C11. It was designed around a single idea: the data path should be as fast as the hardware allows, with no unnecessary work anywhere between file and socket.
30
31	In practice, this means using `sendfile` where the OS supports it, keeping the hot path free of allocations, and handling TCP backpressure correctly so the pipe never stalls under load. The result is an FTP daemon that saturates a full Gigabit Ethernet link in both directions — on Linux, macOS, or any POSIX-compliant system — without any client-side tuning.
32
33	The same binary model also targets PS4 and PS5 as console payloads, with on-screen notifications and an optional browser-based file explorer. This is an extension of the same codebase, not a fork — the POSIX foundation is identical.
34
35	```
36	Philosophy
37	├── Keep the data path fast → sendfile fast path, zero-copy where available
38	├── Handle TCP correctly → partial sends, EINTR, backpressure-aware buffers
39	├── Stay portable → C11, POSIX, standard toolchain
40	├── Be predictable under load → no dynamic allocation per transfer, no surprises
41	└── Extras are opt-in → encryption, rate limiting, web UI — all compile-time
42	```
43
44	---
45
46	## ⚡ Performance
47
48	> `zftpd` saturates a full Gigabit Ethernet link — ~112 MB/s sustained in both directions.
49
50	This is the physical ceiling of a 1 GbE connection. It is achieved out of the box, with no kernel tuning required.
51
52	```
53	Benchmark — single stream, wired 1 GbE, plain transfer
54
55	Download ████████████████████████████████████████████ 112 MB/s
56	Upload █████████████████████████████████████████ 108 MB/s
57	────────
58	Physical ceiling (1 GbE) 125 MB/s
59	```
60
61	What makes it fast:
62
63	\| Technique \| What it does \|
64	\|---\|---\|
65	\| `sendfile` kernel fast path \| Moves file data directly to the socket — zero userspace copies \|
66	\| Partial-send loop \| Handles short writes without stalling or corrupting the stream \|
67	\| EINTR-safe I/O \| Signal interrupts are absorbed cleanly in the hot loop \|
68	\| Allocation-free transfer path \| No `malloc`, no locking per packet or per transfer \|
69	\| Token-bucket limiter is opt-in \| Adds zero overhead when rate limiting is not needed \|
70
71	> On encryption: enabling `AUTH XCRYPT` (ChaCha20) disables `sendfile` and switches to buffered I/O. Throughput becomes CPU-bound. For maximum speed on a trusted network, use plain transfers — that is what `sendfile` is there for.
72
73	---
74
75	## ✦ Features
76
77	<table>
78	<tr>
79	<td width="50%" valign="top">
80
81	Transfer engine
82	- `sendfile` zero-copy fast path (Linux · BSD · macOS)
83	- Fallback to buffered I/O when encrypted
84	- Backpressure-aware send loop, EINTR-safe
85	- Upload resume: `REST` + `STOR`
86	- Append mode: `APPE`
87	- Server-side copy: `CPFR`/`CPTO`, `COPY` (async background thread)
88	- Cross-device move: `RNTO` fallback with async copy
89	- Transfer rate limiting via token bucket (compile-time, opt-in)
90
91	Connection handling
92	- Active mode: `PORT`
93	- Passive mode: `PASV`, `EPSV`
94	- Control and data channel timeouts
95	- Session idle timeout
96	- Up to `FTP_MAX_SESSIONS` concurrent sessions
97
98	</td>
99	<td width="50%" valign="top">
100
101	Security
102	- Path canonicalization — no traversal possible
103	- Optional blocklist for `/dev`, `/proc`, `/sys`
104	- Optional ChaCha20 stream cipher with PSK (`AUTH XCRYPT`)
105
106	Observability
107	- Structured per-session logging
108	- Transfer stats: bytes sent/received, files transferred
109	- Per-command logging (compile-time toggle)
110
111	Platform extras
112	- Linux, macOS, PS4, PS5
113	- On-screen IP/port notification on PS4 and PS5
114	- ZHTTP web file explorer (compile-time, see [ZHTTP](#-zhttp))
115
116	</td>
117	</tr>
118	</table>
119
120	<details>
121	<summary><b>Complete FTP command reference</b></summary>
122
123	<br/>
124
125	\| Group \| Commands \|
126	\|---\|---\|
127	\| Authentication \| `USER` `PASS` `QUIT` `NOOP` \|
128	\| Navigation \| `CWD` `CDUP` `PWD` \|
129	\| Directory listing \| `LIST` `NLST` `MLSD` `MLST` \|
130	\| File transfer \| `RETR` `STOR` `APPE` `REST` \|
131	\| File management \| `DELE` `RMD` `MKD` `RNFR` `RNTO` \|
132	\| Server-side copy \| `CPFR` `CPTO` `COPY` — async background thread \|
133	\| Data connection \| `PORT` `PASV` `EPSV` \|
134	\| Metadata \| `SIZE` `MDTM` `STAT` `SYST` `FEAT` `HELP` \|
135	\| Transfer parameters \| `TYPE` `MODE` `STRU` \|
136	\| Negotiation \| `OPTS` `CLNT` \|
137	\| Site extensions \| `SITE CHMOD` \|
138	\| Encryption \| `AUTH XCRYPT` — ChaCha20 with PSK (opt-in) \|
139
140	</details>
141
142	---
143
144	## 🛠 Best Setup
145
146	### Network — wired is the only choice
147
148	`zftpd` performs at the physical limit of your network. The bottleneck is almost always the medium, not the software. Wi-Fi is the bottleneck — even Wi-Fi 6 introduces retransmissions and variable latency that collapse sustained FTP throughput. Use a wired connection.
149
150	The optimal topology is a direct Ethernet cable between source and destination, eliminating every unnecessary hop:
151
152	```
153	[Source machine]
154	│
155	Ethernet cable
156	│
157	[Destination machine]
158	```
159
160	If a switch is needed, any Gigabit switch works. Avoid powerline adapters and MoCA bridges — they introduce jitter that disrupts sustained transfers.
161
162	Assign static IPs on both ends (e.g. `192.168.100.1` / `192.168.100.2`). This removes DHCP latency and keeps the setup fully deterministic.
163
164	---
165
166	### FTP clients
167
168	\| Client \| Platform \| Recommendation \|
169	\|---\|---\|---\|
170	\| FileZilla \| Windows · macOS · Linux \| Best general-purpose choice. Enable parallel transfers for directory trees. \|
171	\| WinSCP \| Windows \| Excellent throughput and error recovery. \|
172	\| lftp \| Linux · macOS \| Best CLI option. `pget -n 4` enables parallel chunked downloads. \|
173	\| Cyberduck \| macOS · Windows \| Solid for occasional transfers. \|
174	\| OS built-in FTP \| any \| ❌ Avoid — artificially capped speeds, no resume support. \|
175
176	Things that matter on the client side:
177
178	- Transfer mode must be Binary (`TYPE I`). `zftpd` defaults to Binary, but a misconfigured client can override this silently — always verify.
179	- Use passive mode (`PASV`). It's the default and works cleanly behind NAT and firewalls. Active mode requires the server to reach back to the client and is frequently blocked.
180	- Enable parallel connections for large directory trees. FileZilla exposes this under Site Manager → Transfer Settings. It will not increase single-file speed, but dramatically reduces total time for many small files.
181	- Disable client-side CRC or integrity checks if offered. TCP guarantees delivery; checksumming again adds latency for no benefit.
182
183	---
184
185	### Linux — optional kernel tuning
186
187	`zftpd` reaches full Gigabit speed with default kernel parameters. If you are feeding a very fast NVMe drive into the network and want to raise the ceiling further:
188
189	```bash
190	# Raise socket buffer limits — run as root, optional
191	sysctl -w net.core.rmem_max=134217728
192	sysctl -w net.core.wmem_max=134217728
193	sysctl -w net.ipv4.tcp_rmem="4096 87380 134217728"
194	sysctl -w net.ipv4.tcp_wmem="4096 65536 134217728"
195	```
196
197	To make these persistent, add them to `/etc/sysctl.conf`.
198
199	<details>
200	<summary><b>PS4 / PS5 — console-specific notes</b></summary>
201
202	<br/>
203
204	- `zftpd` requires a payload loader to run on console (WebKit / PPPwn / GoldHEN on PS4; etaHEN or equivalent on PS5). The daemon itself does not require a resident HEN.
205	- Launch after the system is fully booted and the loader is ready. The on-screen notification will display the IP and port.
206	- For maximum throughput: direct cable from console to PC, static IPs, no router in between.
207	- Avoid initiating transfers while background downloads or system updates are active — the network stack is shared.
208	- If you see "payload already loaded": a previous instance is still active. `zftpd` will attempt to terminate it and restart on the default port. If that fails, it tries up to 9 subsequent ports (`FTP_DEFAULT_PORT+1` … `+9`).
209
210	</details>
211
212	</details>
213
214	<details>
215	<summary><b>PS5 — firmware-dependent transfer speed</b></summary>
216
217	<br/>
218
219	Transfer speed to the internal storage (`/data/...`) varies significantly across PS5 firmware versions. This is not a `zftpd` limitation — it is caused by Sony's kernel-level I/O driver improvements across firmware updates.
220
221	```
222	Upload speed to internal storage (wired 1 GbE, measured across multiple consoles)
223
224	FW 4.03 ████████ ~20 MB/s
225	FW 8.60 ████████████████████ ~50 MB/s
226	FW 9.00 ██████████████████████████████████ ~85 MB/s
227	FW 10.00 ████████████████████████████████████████████ ~113 MB/s
228	────────
229	Physical ceiling (1 GbE) 125 MB/s
230	```
231
232	Why it happens:
233	- The PS5 internal storage uses the PFS (PlayStation File System) with mandatory block-level encryption. Every `write()` syscall goes through the kernel's crypto + NVMe pipeline.
234	- Sony has incrementally improved this pipeline (write scheduling, page cache, NVMe queue depth) across firmware releases. On FW 10.00 the kernel saturates Gigabit.
235	- This is not related to PFS crypto cost alone — if it were, speeds would be constant across all firmware. The scaling pattern proves the bottleneck is kernel I/O scheduling, not encryption.
236
237	External USB storage (`/mnt/usb0/...`, typically exFAT) bypasses PFS entirely and consistently reaches ~113 MB/s regardless of firmware version.
238
239	What this means for users:
240	- On older firmware (< 9.00), internal storage writes are kernel-limited. No FTP server (zftpd, ftpsrv, GoldHEN) can exceed these speeds — the limit is in the OS.
241	- For maximum speed on older firmware, transfer to external USB-C storage instead.
242	- On FW 9.00+ the internal storage speed approaches Gigabit saturation.
243
244	</details>
245
246	---
247
248	## 📦 Build & commands
249
250	Make targets (host auto-detection, best-effort toolchains):
251
252	- `make` — build default target (Linux on Linux host, macOS on macOS host) release.
253	- `make release-all` — release build per platform detected (macos, linux, ps3, ps4, ps5).
254	- `make debug-all` — debug build per platform detected.
255	- `make release-matrix` — release build per platform and variant `ENABLE_ZHTTPD=0/1`, producing ELF/BIN dove applicabile.
256	- `make TARGET=<platform> BUILD_TYPE=<release\|debug> [ENABLE_ZHTTPD=0\|1] clean all` — build singolo.
257
258	Useful Makefile variables:
259
260	- `TARGET`: `linux`, `macos`, `ps3`, `ps4`, `ps5` (auto su host). Case-insensitive.
261	- `BUILD_TYPE`: `release` (default), `debug`.
262	- `ENABLE_ZHTTPD`: `1` abilita web UI zhttp (default 0 su console, 1 su PC). Influenza naming: es. `zftpd-ps5-zhttp-v1.3.0.bin`.
263	- `ARTIFACT_PREFIX`: prefisso binari (default `zftpd`).
264	- `ffi_langs`: opzionale, per build dei binding FFI.
265
266	Output naming (release):
267	- ELF: `build/<target>/release[/ -zhttp]/zftpd-<platform-tag>[-zhttp]-v<version>.elf`
268	- BIN (console): `... .bin`
269
270	Execution (binaries host):
271
272	```
273	./build/macos/release/zftpd-macos-$(uname -m)-v1.3.0 -p <port> -d <root>
274	./build/linux/release/zftpd-linux-$(uname -m)-v1.3.0.elf -p <port> -d <root>
275	```
276
277	Supported options:
278	- `-p <PORT>` (default 2121)
279	- `-d <DIR>` root FTP
280	- `-h` help
281
282
283	---
284
285	## 📦 Build
286
287	Output artifacts are versioned and platform-tagged, placed in `build/<target>/<build_type>/`.
288
289	### Requirements
290
291	\| \| \|
292	\|---\|---\|
293	\| Compiler \| C11 — `gcc` or `clang` \|
294	\| Build system \| `make` \|
295	\| `.bin` generation \| `objcopy` (binutils or llvm-objcopy); PS4: `orbis-objcopy`; PS5: `prospero-objcopy` \|
296	\| PS4 \| `PS4_PAYLOAD_SDK` set in environment \|
297	\| PS5 \| `PS5_PAYLOAD_SDK` set in environment \|
298
299	### Commands
300
301	```bash
302	# Targets
303	make TARGET=linux
304	make TARGET=macos
305	make TARGET=ps4
306	make TARGET=ps5
307
308	# Modifiers
309	make TARGET=linux BUILD_TYPE=debug
310	make TARGET=linux ENABLE_ZHTTPD=1 # enable web UI (off by default on POSIX)
311	make TARGET=ps5 ENABLE_ZHTTPD=0 # disable web UI (on by default on console)
312
313	# Tests (POSIX only)
314	make TARGET=linux test
315	make TARGET=macos test
316	```
317
318	### Artifacts
319
320	\| Platform \| Output \|
321	\|---\|---\|
322	\| Linux \| `build/linux/release/zftpd-linux-<arch>-v<ver>.elf` \|
323	\| macOS \| `build/macos/release/zftpd-macos-<arch>-v<ver>` \|
324	\| PS4 \| `zftpd-ps4-v<ver>.bin` · `zftpd-ps4-v<ver>.elf` \|
325	\| PS5 \| `zftpd-ps5-v<ver>.bin` · `zftpd-ps5-v<ver>.elf` \|
326
327	---
328
329	## 🚀 Running
330
331	### Linux
332
333	```bash
334	./build/linux/release/zftpd-linux-<arch>-v<version>.elf [-p <port>] [-d <root>]
335	```
336
337	### macOS
338
339	```bash
340	./build/macos/release/zftpd-macos-<arch>-v<version> [-p <port>] [-d <root>]
341	```
342
343	### PS4
344
345	Send `.bin` to your payload loader, or `.elf` if the loader accepts ELF directly.
346	On startup: on-screen notification displays IP and port.
347
348	### PS5
349
350	Send `.bin` or `.elf` depending on your loader.
351	On startup: `FTP: <ip>:<port>` notification.
352
353	---
354
355	## ⚙️ Configuration
356
357	All configuration is compile-time, in [`include/ftp_config.h`](include/ftp_config.h).
358
359	\| Macro \| Default \| Notes \|
360	\|---\|---\|---\|
361	\| `FTP_DEFAULT_PORT` \| `2121` (POSIX) · `2122` (console) \| Listening port \|
362	\| `FTP_MAX_SESSIONS` \| — \| Maximum concurrent client sessions \|
363	\| `FTP_SESSION_TIMEOUT` \| — \| Idle session timeout \|
364	\| `FTP_TRANSFER_RATE_LIMIT_BPS` \| disabled \| Token-bucket average rate cap \|
365	\| `FTP_TRANSFER_RATE_BURST_BYTES` \| disabled \| Token-bucket burst allowance \|
366	\| `FTP_LOG_COMMANDS` \| — \| Log every received command \|
367
368	---
369
370	## 🌐 ZHTTP
371
372	ZHTTP is a lightweight HTTP server embedded in `zftpd` that serves a browser-based file explorer. It allows browsing, downloading, and optionally uploading files from any browser on the local network — no FTP client required.
373
374	\| Target \| Default \| To override \|
375	\|---\|---\|---\|
376	\| PS4 / PS5 \| ✅ on \| `make TARGET=ps5 ENABLE_ZHTTPD=0` \|
377	\| Linux / macOS \| ❌ off \| `make TARGET=linux ENABLE_ZHTTPD=1` \|
378
379	Once the daemon is running, open `http://<ip>:<port>/` — the HTTP port mirrors the configured FTP port.
380
381	Upload support is enabled automatically alongside ZHTTP (`ENABLE_WEB_UPLOAD=1`).
382
383	> Security: ZHTTP has no authentication beyond network access. It is designed for local-network use. Do not expose it on a public interface.
384
385	---
386
387	## Acknowledgements
388
389	I would like to express our sincere thanks to:
390
391	- hippie68 — for the PS4 FTP reference implementation
392	- John Törnblom — for the PS5 payload framework
393	- Drakmor — for the inspiration in the implementation of PFR / CPTO / COPY
394	- The PlayStation homebrew community — for testing, feedback, and ongoing support
395
396	---
397
398	<div align="center">
399
400	Released under the [MIT License](LICENSE)
401
402	</div>
403

Seregon/zftpd