Add a reorderpcap tool

This commit is contained in:
2026-10-05 20:35:47 +02:00
parent c11c893845
commit e70549337e
5 changed files with 46 additions and 28 deletions
+30 -1
View File
@@ -17,6 +17,12 @@ Flags:
Uploads larger than 1 MB are spooled to temporary files under `$TMPDIR` (default `/tmp`) rather than held in memory. The temporary files are removed after each request.
If `/tmp` is a RAM-backed tmpfs, as on Raspberry Pi OS, uploads still use RAM, and a file bigger than the tmpfs fails. In that case point `TMPDIR` at a disk directory:
```sh
TMPDIR=/path/on/disk ./pcap-web-tools -max-upload-mb 5000
```
## Tools
### Hex → pcap (`/hex2pcap`)
@@ -54,12 +60,35 @@ Splits an uploaded pcap file into parts with at most N packets or at most N KB/M
- The upload is checked completely before the zip is sent, so errors show on the page instead of as a broken download.
- Only classic pcap is supported. Convert pcapng first with `editcap -F pcap in.pcapng out.pcap`.
### Reorder pcap (`/reorder`)
Sorts the packets of an uploaded pcap file by timestamp and returns the result as `<name>_reordered.pcap`. The output is byte-identical to `reordercap`'s.
- The sort is stable, so packets with equal timestamps keep their original order.
- The file header and packet records are copied unchanged.
- The response headers report the number of packets (`X-Pcap-Packets`), the number of packets that were out of order (`X-Pcap-Out-Of-Order`) and whether a truncated final packet was dropped (`X-Pcap-Truncated`). These are useful when scripting with `curl -F file=@in.pcap`.
How it works:
- **Index:** one sequential pass keeps a 16-byte index entry per packet (timestamp, plus offset and length), which is then sorted.
- **Write:** the packets are copied out in sorted order using 16 parallel reads in 8 MB batches, reading the next batch while the previous one is written. SSDs are much faster at random reads with many requests in flight.
- **Already sorted:** if the file is already in order, it is copied straight through.
Measured on a Raspberry Pi 5 with NVMe, for 3 million 1500-byte packets (4.5 GB) with fully random timestamps, the worst case:
| | Time | Peak memory |
|---|---|---|
| Server: upload, reorder, download over localhost | 95 s | 108 MB |
| `reordercap` | about 3.5 h (extrapolated from 0.35 MB/s) | 180 MB |
Memory is roughly 16 bytes per packet for the index, plus about 16 MB of copy buffers and Go GC headroom. Captures that are mostly in order, the usual case, read nearly sequentially and are faster.
## Layout
```
main.go entry point, HTTP server
internal/web/ handlers, tool registry, embedded templates and CSS
internal/pcap/ pcap file reader, writer and splitter
internal/pcap/ pcap file reader, writer, splitter and reorderer
internal/inet/ IPv4/IPv6 length and checksum fixups, IP header locator
internal/hexutil/ lenient hex decoding
```