22 KiB
22 KiB
phase, plan, type, wave, depends_on, files_modified, autonomous, requirements, must_haves
| phase | plan | type | wave | depends_on | files_modified | autonomous | requirements | must_haves | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 01-data-pipeline | 01 | execute | 1 |
|
true |
|
|
Purpose: Create the foundational project structure that all subsequent plans build on. The workspace must compile, the eBPF build pipeline must work, shared types must be defined for kernel-userspace communication, and the platform trait must be ready for the Linux collector implementation.
Output: Compiling workspace with three crates (userspace, ebpf, common), working eBPF build pipeline, privilege checking, platform trait, and all shared data types.
<execution_context> @/Users/zrowitsch/local_src/tcptop/.claude/get-shit-done/workflows/execute-plan.md @/Users/zrowitsch/local_src/tcptop/.claude/get-shit-done/templates/summary.md </execution_context>
@.planning/PROJECT.md @.planning/ROADMAP.md @.planning/STATE.md @.planning/phases/01-data-pipeline/01-CONTEXT.md @.planning/phases/01-data-pipeline/01-RESEARCH.md Task 1: Create Aya workspace with shared types and eBPF build pipeline Cargo.toml, rust-toolchain.toml, .cargo/config.toml, tcptop/Cargo.toml, tcptop/build.rs, tcptop/src/main.rs, tcptop-common/Cargo.toml, tcptop-common/src/lib.rs, tcptop-ebpf/Cargo.toml, tcptop-ebpf/src/main.rs .planning/phases/01-data-pipeline/01-RESEARCH.md (architecture patterns, code examples, version numbers), CLAUDE.md (technology stack, version compatibility) Create the full Aya eBPF workspace with three crates. This is a greenfield project -- no existing code.**rust-toolchain.toml:**
Pin nightly channel (use `nightly-2026-01-15` or a recent stable nightly). Include components: `rust-src`, `rustfmt`, `clippy`. Add `bpfel-unknown-none` as a target.
**.cargo/config.toml:**
Add any needed build flags for the eBPF target (if aya-build handles this automatically, this file may be minimal or empty -- check aya-build behavior).
**Cargo.toml (workspace root):**
```toml
[workspace]
members = ["tcptop", "tcptop-common", "tcptop-ebpf"]
resolver = "2"
[workspace.dependencies]
aya = { version = "0.13.1", features = ["async_tokio"] }
aya-log = "0.2"
tokio = { version = "1", features = ["full"] }
anyhow = "1"
thiserror = "2"
clap = { version = "4.6", features = ["derive"] }
log = "0.4"
env_logger = "0.11"
nix = { version = "0.29", features = ["user"] }
procfs = "0.18"
signal-hook = "0.3"
```
**tcptop-common/Cargo.toml:**
```toml
[package]
name = "tcptop-common"
version = "0.1.0"
edition = "2021"
[features]
default = []
user = ["no-std-compat-off"] # feature flag if needed
[dependencies]
# No dependencies -- this crate must be no_std compatible
```
The common crate must compile under both `no_std` (for eBPF) and `std` (for userspace).
**tcptop-common/src/lib.rs:**
Define ALL shared types used between kernel and userspace. Every struct MUST be `#[repr(C)]` with only primitive fields (no String, Vec, Option, enums with data larger than simple discriminants).
**LOCKED DECISION -- Use `union` for TcptopEventData.** The `#[repr(C)] pub union TcptopEventData` approach is the canonical layout. Plan 02's `parse_event` uses `unsafe { &event.data.data_event }` and `unsafe { &event.data.state_event }` accessors keyed on `event_type`. Do NOT use a flat struct alternative -- the union is the contract between Plans 01 and 02.
Types to define:
```rust
#![no_std]
// IP address family
pub const AF_INET: u16 = 2;
pub const AF_INET6: u16 = 10;
#[repr(C)]
#[derive(Clone, Copy)]
pub struct DataEvent {
pub pid: u32,
pub comm: [u8; 16], // process name from bpf_get_current_comm
pub af_family: u16, // AF_INET or AF_INET6
pub sport: u16,
pub dport: u16,
pub _pad: u16, // alignment padding
pub saddr: [u8; 16], // IPv4 in first 4 bytes, or full IPv6
pub daddr: [u8; 16],
pub bytes: u32, // bytes transferred in this call
pub srtt_us: u32, // smoothed RTT (shifted <<3 from kernel, we store raw)
}
#[repr(C)]
#[derive(Clone, Copy)]
pub struct StateEvent {
pub pid: u32,
pub af_family: u16,
pub sport: u16,
pub dport: u16,
pub _pad: u16,
pub saddr: [u8; 16],
pub daddr: [u8; 16],
pub old_state: u32,
pub new_state: u32,
}
// Event tag for the ring buffer -- simple discriminant
pub const EVENT_TCP_SEND: u32 = 1;
pub const EVENT_TCP_RECV: u32 = 2;
pub const EVENT_UDP_SEND: u32 = 3;
pub const EVENT_UDP_RECV: u32 = 4;
pub const EVENT_TCP_STATE: u32 = 5;
#[repr(C)]
#[derive(Clone, Copy)]
pub struct TcptopEvent {
pub event_type: u32, // one of EVENT_* constants
pub _pad: u32, // alignment to 8 bytes
pub data: TcptopEventData,
}
// Tagged union -- event_type discriminant tells which variant to read.
// Both DataEvent and StateEvent must fit. DataEvent is larger.
// Userspace reads via: unsafe { &event.data.data_event } or unsafe { &event.data.state_event }
#[repr(C)]
#[derive(Clone, Copy)]
pub union TcptopEventData {
pub data_event: DataEvent,
pub state_event: StateEvent,
}
```
If the eBPF crate fails to compile with `union` under `no_std` + `bpfel-unknown-none`, investigate the specific error (likely a missing `Copy` bound or alignment issue) and fix it while preserving the union layout. The union is the locked contract -- do not fall back to a flat struct.
**tcptop-ebpf/Cargo.toml:**
```toml
[package]
name = "tcptop-ebpf"
version = "0.1.0"
edition = "2021"
[dependencies]
aya-ebpf = "0.1"
aya-log-ebpf = "0.1"
tcptop-common = { path = "../tcptop-common" }
[[bin]]
name = "tcptop"
path = "src/main.rs"
```
**tcptop-ebpf/src/main.rs:**
Minimal skeleton that compiles:
```rust
#![no_std]
#![no_main]
use aya_ebpf::{macros::map, maps::RingBuf};
#[map]
static EVENTS: RingBuf = RingBuf::with_byte_size(256 * 1024, 0);
// Placeholder -- kprobes added in Plan 02
#[panic_handler]
fn panic(_info: &core::panic::PanicInfo) -> ! {
loop {}
}
```
Note: aya-ebpf may provide its own panic handler. If compilation fails due to duplicate panic handlers, remove the manual one.
**tcptop/Cargo.toml:**
```toml
[package]
name = "tcptop"
version = "0.1.0"
edition = "2021"
build = "build.rs"
[dependencies]
aya = { workspace = true }
aya-log = { workspace = true }
tokio = { workspace = true }
anyhow = { workspace = true }
thiserror = { workspace = true }
clap = { workspace = true }
log = { workspace = true }
env_logger = { workspace = true }
nix = { workspace = true }
procfs = { workspace = true }
signal-hook = { workspace = true }
tcptop-common = { path = "../tcptop-common" }
[build-dependencies]
aya-build = "0.1"
```
**tcptop/build.rs:**
Use aya-build to compile the eBPF crate. Based on research:
```rust
fn main() {
aya_build::build_ebpf(&["tcptop-ebpf"])
.expect("Failed to build eBPF programs");
}
```
If the exact API differs, consult aya-build 0.1.3 docs and adapt.
**tcptop/src/main.rs:**
Minimal entry point that calls privilege check and exits:
```rust
mod privilege;
mod collector;
mod model;
fn main() {
privilege::check_privileges();
println!("tcptop: privilege check passed, eBPF loading not yet implemented");
}
```
After creating all files, run `cargo check` (not `cargo build` -- building eBPF requires bpf-linker which may not be installed). If `cargo check` fails on the eBPF crate, that is acceptable -- focus on the userspace crate compiling: `cargo check -p tcptop --lib` or similar.
**tcptop/src/privilege.rs (per D-09, D-10, D-11):**
```rust
use nix::unistd::geteuid;
use std::process;
pub fn check_privileges() {
if geteuid().is_root() {
return;
}
if has_required_capabilities() {
return;
}
eprintln!("error: tcptop requires root privileges. Run with sudo.");
process::exit(77);
}
fn has_required_capabilities() -> bool {
let status = match std::fs::read_to_string("/proc/self/status") {
Ok(s) => s,
Err(_) => return false,
};
for line in status.lines() {
if let Some(hex) = line.strip_prefix("CapEff:") {
let hex = hex.trim();
if let Ok(caps) = u64::from_str_radix(hex, 16) {
let cap_perfmon = 1u64 << 38;
let cap_bpf = 1u64 << 39;
return (caps & cap_perfmon != 0) && (caps & cap_bpf != 0);
}
}
}
false
}
```
**tcptop/src/model.rs (per D-05, D-06, D-07, D-08, D-12, D-15):**
Define the userspace-side connection model. These are NOT the eBPF shared types (those are in tcptop-common). These are rich Rust types used by the aggregator and output formatter.
```rust
use std::net::IpAddr;
use std::time::Instant;
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub enum Protocol {
Tcp,
Udp,
}
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct ConnectionKey {
pub protocol: Protocol,
pub local_addr: IpAddr,
pub local_port: u16,
pub remote_addr: IpAddr,
pub remote_port: u16,
}
#[derive(Debug, Clone)]
pub struct ConnectionRecord {
pub key: ConnectionKey,
pub pid: u32,
pub process_name: String,
pub tcp_state: Option<TcpState>, // None for UDP (per D-07)
pub bytes_in: u64,
pub bytes_out: u64,
pub packets_in: u64,
pub packets_out: u64,
pub rate_in: f64, // bytes/sec (per DATA-06)
pub rate_out: f64, // bytes/sec
pub prev_bytes_in: u64, // for rate calculation
pub prev_bytes_out: u64,
pub rtt_us: Option<u32>, // microseconds, None for UDP
pub last_seen: Instant,
pub is_partial: bool, // true for pre-existing connections (per D-15)
pub is_closed: bool, // true when TCP state -> CLOSE (per D-12)
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum TcpState {
Established,
SynSent,
SynRecv,
FinWait1,
FinWait2,
TimeWait,
Close,
CloseWait,
LastAck,
Listen,
Closing,
NewSynRecv,
}
impl TcpState {
pub fn from_kernel(state: u32) -> Option<Self> {
// Kernel TCP state values from include/net/tcp_states.h
match state {
1 => Some(TcpState::Established),
2 => Some(TcpState::SynSent),
3 => Some(TcpState::SynRecv),
4 => Some(TcpState::FinWait1),
5 => Some(TcpState::FinWait2),
6 => Some(TcpState::TimeWait),
7 => Some(TcpState::Close),
8 => Some(TcpState::CloseWait),
9 => Some(TcpState::LastAck),
10 => Some(TcpState::Listen),
11 => Some(TcpState::Closing),
12 => Some(TcpState::NewSynRecv),
_ => None,
}
}
pub fn as_str(&self) -> &'static str {
match self {
TcpState::Established => "ESTABLISHED",
TcpState::SynSent => "SYN_SENT",
TcpState::SynRecv => "SYN_RECV",
TcpState::FinWait1 => "FIN_WAIT1",
TcpState::FinWait2 => "FIN_WAIT2",
TcpState::TimeWait => "TIME_WAIT",
TcpState::Close => "CLOSE",
TcpState::CloseWait => "CLOSE_WAIT",
TcpState::LastAck => "LAST_ACK",
TcpState::Listen => "LISTEN",
TcpState::Closing => "CLOSING",
TcpState::NewSynRecv => "NEW_SYN_RECV",
}
}
}
impl std::fmt::Display for TcpState {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "{}", self.as_str())
}
}
```
**tcptop/src/collector/mod.rs (per PLAT-03):**
Define the platform abstraction trait. Use `tokio::sync::mpsc` channel pattern from research.
```rust
pub mod linux; // Will be implemented in Plan 02
use crate::model::ConnectionRecord;
use anyhow::Result;
use tokio::sync::mpsc;
/// Events emitted by the collector to the aggregator
#[derive(Debug)]
pub enum CollectorEvent {
TcpSend { key: crate::model::ConnectionKey, pid: u32, comm: String, bytes: u32, srtt_us: u32 },
TcpRecv { key: crate::model::ConnectionKey, pid: u32, comm: String, bytes: u32, srtt_us: u32 },
UdpSend { key: crate::model::ConnectionKey, pid: u32, comm: String, bytes: u32 },
UdpRecv { key: crate::model::ConnectionKey, pid: u32, comm: String, bytes: u32 },
TcpStateChange { key: crate::model::ConnectionKey, pid: u32, old_state: u32, new_state: u32 },
}
/// Platform abstraction for network data collection.
/// Linux: eBPF kprobes + tracepoints (Phase 1)
/// macOS: libpcap + PKTAP (Phase 4)
#[async_trait::async_trait]
pub trait NetworkCollector: Send {
/// Start collecting network events. Sends events to the provided channel.
/// Returns when stop() is called or an error occurs.
async fn start(&mut self, tx: mpsc::Sender<CollectorEvent>) -> Result<()>;
/// Stop collecting and clean up kernel resources (detach probes, etc).
async fn stop(&mut self) -> Result<()>;
/// Bootstrap pre-existing connections (e.g., from /proc/net/tcp).
/// Called once before start().
fn bootstrap_existing(&self) -> Result<Vec<ConnectionRecord>>;
}
```
Add `async-trait = "0.1"` to tcptop/Cargo.toml dependencies if not already present.
Create a placeholder `tcptop/src/collector/linux.rs` with a comment: `// Linux eBPF collector -- implemented in Plan 02`
**Update tcptop/src/main.rs:**
Wire up the privilege check and declare all modules:
```rust
mod privilege;
mod collector;
mod model;
fn main() {
privilege::check_privileges();
println!("tcptop: privilege check passed");
// eBPF loading and event loop implemented in Plan 02 and Plan 03
}
```
<success_criteria>
- Workspace structure with three crates exists and userspace + common crates compile
- All shared types are #[repr(C)] with IPv6-ready [u8; 16] address fields
- TcptopEventData uses union layout (locked contract for Plan 02 parse_event)
- Privilege check exits 77 with exact error message per D-09
- NetworkCollector trait is defined with start/stop/bootstrap_existing per PLAT-03
- ConnectionRecord has all fields needed for DATA-01 through DATA-07
- eBPF crate has skeleton with RingBuf map declaration </success_criteria>