diff --git a/PLAN.md b/PLAN.md index 5d72151..3d19954 100644 --- a/PLAN.md +++ b/PLAN.md @@ -1887,6 +1887,8 @@ Editor-to-filesystem integration is complete for the initial C workflow. The bro Compiler source diagnostics now have a product UI path. Clang-format stderr records are parsed into path, line, column, severity, and message entries while the complete raw terminal stream remains available. Selecting a `/project/main.c` diagnostic focuses the source editor at the bounded reported position. A deterministic checked-in AArch64 fixture emits the same source-location format through the real standard-error syscall path; native and Chromium regressions verify its nonzero exit, structured display, and exact editor selection. Remaining Phase 6 work is direct child-process and pipe semantics, build cache policy, and interactive-performance improvements. +The first pipe checkpoint adds AArch64 `pipe2` with process-owned descriptors and a fixed 64 KiB FIFO queue. The initial contract deliberately requires `O_NONBLOCK`: reads distinguish `EAGAIN` from EOF, writes are partial at remaining capacity and return `EPIPE` after the reader closes, `fstat` identifies FIFO descriptors, `lseek` returns `ESPIPE`, close-on-exec shares the ordinary descriptor flag path, and pipe ends count against the open-file limit. A checked-in assembly fixture round-trips bytes through the pipe and passes on the native host and in Chromium. Blocking pipe suspension/wakeup and descriptor sharing across concurrent child processes remain the next orchestration checkpoint; build cache policy and interactive-performance work also remain. + Do not begin the full web IDE before item 30 passes. --- diff --git a/README.md b/README.md index 7ba7f49..39947ae 100644 --- a/README.md +++ b/README.md @@ -71,7 +71,7 @@ The CLI writes this snapshot after execution stops even when a resource limit produces a diagnostic, so bounded compiler runs can retain their cache and other completed filesystem mutations for a later run. -The runnable instruction/syscall profile remains fixture-driven. The current static Rust program adds single-thread atomics and barriers, 128-bit vector moves/stores, byte popcount and reduction, multiplication/division, and signed shifts to the earlier libc instruction path. The runtime implements `openat`, `close`, `lseek`, file `read`/`write`, polling, deterministic process/signal setup, static-ELF `execve`, anonymous memory management, terminal output, and exit. Invalid guest arguments return Linux errno values; instruction, syscall, committed-memory, ephemeral-filesystem, open-file, and combined-output limits are enforced before host side effects. +The runnable instruction/syscall profile remains fixture-driven. The current static Rust program adds single-thread atomics and barriers, 128-bit vector moves/stores, byte popcount and reduction, multiplication/division, and signed shifts to the earlier libc instruction path. The runtime implements `openat`, `close`, `lseek`, file `read`/`write`, bounded nonblocking `pipe2`, polling, deterministic process/signal setup, static-ELF `execve`, anonymous memory management, terminal output, and exit. Invalid guest arguments return Linux errno values; instruction, syscall, committed-memory, ephemeral-filesystem, open-file, and combined-output limits are enforced before host side effects. The native CPython checkpoint is reproducible without committing its large generated artifacts. `guest-tests/cpython/build.sh` creates a statically linked @@ -86,12 +86,13 @@ The verified multi-file and third-party imports, package-image overlays, traceback behavior, and current limitations are summarized in [`docs/cpython-compatibility.md`](docs/cpython-compatibility.md). -Phase 6 now has its first process-orchestration primitive. The checked-in -`guest-tests/execve-launcher` parent replaces itself with a second static ELF -loaded from an installable project image. Native and Chromium regressions -verify rebuilt arguments and environment plus ordinary and close-on-exec file -descriptors. Concurrent children, pipes, and the Clang/LLD/musl package remain -the next compilation milestones. +Phase 6 includes static process replacement, a packaged Clang/LLD/musl +toolchain, browser-edited C source, linked compiler diagnostics, and the first +pipe primitive. The checked-in `guest-tests/execve-launcher` replaces itself +with a second static ELF loaded from a project image, while +`guest-tests/pipe-roundtrip` exercises a fixed-capacity `O_NONBLOCK` pipe in +both native and Chromium hosts. Concurrent children and blocking pipe +scheduling remain the next process-orchestration milestone. The browser build generates its Memory64, JSPI, and P-code `.wasm` probes before starting Vite. Generated artifacts are not committed. Select **Uploaded AArch64 ELF** to run an external static executable with a chosen `argv[0]` and one argument per line; the executable is transferred directly to the runtime Worker. diff --git a/crates/browser-runtime/src/lib.rs b/crates/browser-runtime/src/lib.rs index 5203d7b..8e28604 100644 --- a/crates/browser-runtime/src/lib.rs +++ b/crates/browser-runtime/src/lib.rs @@ -97,6 +97,8 @@ const PROJECT_PERSISTENCE_WRITE_ELF: &[u8] = include_bytes!( ); const PROJECT_PERSISTENCE_READ_ELF: &[u8] = include_bytes!("../../../guest-tests/project-persistence/project-persistence-read.aarch64.elf"); +const PIPE_ROUNDTRIP_ELF: &[u8] = + include_bytes!("../../../guest-tests/pipe-roundtrip/pipe-roundtrip.aarch64.elf"); const VFS_LIFECYCLE_ELF: &[u8] = include_bytes!("../../../guest-tests/vfs-lifecycle/vfs-lifecycle.aarch64.elf"); const SYSTEM_SERVICES_ELF: &[u8] = @@ -1201,6 +1203,7 @@ fn fixture(name: &str) -> Option<(&'static [u8], &'static [u8])> { "project-persistence-read" => { Some((PROJECT_PERSISTENCE_READ_ELF, b"/project-persistence-read")) } + "pipe-roundtrip" => Some((PIPE_ROUNDTRIP_ELF, b"/pipe-roundtrip")), "vfs-lifecycle" => Some((VFS_LIFECYCLE_ELF, b"/vfs-lifecycle")), "system-services" => Some((SYSTEM_SERVICES_ELF, b"/system-services")), "terminal-input" => Some((TERMINAL_INPUT_ELF, b"/terminal-input")), @@ -1358,6 +1361,25 @@ mod tests { assert_eq!(result.dispatched_syscalls, 2); } + #[test] + fn executes_a_nonblocking_pipe_round_trip() { + let result = execute_fixture( + "pipe-roundtrip", + DEFAULT_INSTRUCTIONS, + DEFAULT_SYSCALLS, + DEFAULT_OUTPUT, + DEFAULT_MEMORY, + DEFAULT_FILESYSTEM, + &[], + ); + + assert_eq!(result.outcome, "exited"); + assert_eq!(result.exit_code, 0); + assert_eq!(result.stdout, "pipe hello\n"); + assert!(result.stderr.is_empty()); + assert_eq!(result.dispatched_syscalls, 7); + } + #[test] fn replaces_a_browser_guest_from_the_filesystem() { let result = execute_fixture( diff --git a/crates/linux-abi/src/lib.rs b/crates/linux-abi/src/lib.rs index a315d48..2643fa1 100644 --- a/crates/linux-abi/src/lib.rs +++ b/crates/linux-abi/src/lib.rs @@ -13,6 +13,7 @@ pub enum Syscall { Ftruncate = 46, Openat = 56, Close = 57, + Pipe2 = 59, Getdents64 = 61, Lseek = 62, Read = 63, @@ -53,6 +54,7 @@ impl Syscall { 46 => Some(Self::Ftruncate), 56 => Some(Self::Openat), 57 => Some(Self::Close), + 59 => Some(Self::Pipe2), 61 => Some(Self::Getdents64), 62 => Some(Self::Lseek), 63 => Some(Self::Read), @@ -96,6 +98,7 @@ pub enum Errno { ArgumentListTooLong = 7, ExecutableFormat = 8, BadFileDescriptor = 9, + TryAgain = 11, OutOfMemory = 12, PermissionDenied = 13, Fault = 14, @@ -106,6 +109,8 @@ pub enum Errno { TooManyOpenFiles = 24, NotTty = 25, NoSpace = 28, + IllegalSeek = 29, + BrokenPipe = 32, Range = 34, NoSystemCall = 38, DirectoryNotEmpty = 39, @@ -134,6 +139,7 @@ mod tests { assert_eq!(Syscall::from_number(46), Some(Syscall::Ftruncate)); assert_eq!(Syscall::from_number(56), Some(Syscall::Openat)); assert_eq!(Syscall::from_number(57), Some(Syscall::Close)); + assert_eq!(Syscall::from_number(59), Some(Syscall::Pipe2)); assert_eq!(Syscall::from_number(61), Some(Syscall::Getdents64)); assert_eq!(Syscall::from_number(62), Some(Syscall::Lseek)); assert_eq!(Syscall::from_number(63), Some(Syscall::Read)); diff --git a/crates/linux-runtime/src/lib.rs b/crates/linux-runtime/src/lib.rs index b038d8d..8941175 100644 --- a/crates/linux-runtime/src/lib.rs +++ b/crates/linux-runtime/src/lib.rs @@ -1,7 +1,7 @@ //! Bounded Linux syscall dispatch for a loaded `AArch64` process. use core::fmt; -use std::collections::BTreeMap; +use std::collections::{BTreeMap, VecDeque}; use binarrow_aarch64::{ Aarch64State, BlockExecutor, BlockProfile, Interpreter, InterpreterInitializationError, @@ -36,6 +36,7 @@ const OPEN_DIRECTORY: u64 = 0x4000; const OPEN_DIRECTORY_FIXTURE_COMPAT: u64 = 0x1_0000; const OPEN_NOFOLLOW: u64 = 0x2_0000; const OPEN_CLOEXEC: u64 = 0x8_0000; +const OPEN_NONBLOCK: u64 = 0x800; const OPEN_PATH: u64 = 0x20_0000; const FCNTL_GET_DESCRIPTOR_FLAGS: u64 = 1; const FCNTL_SET_DESCRIPTOR_FLAGS: u64 = 2; @@ -66,6 +67,7 @@ const STAT_BLOCK_COUNT_OFFSET: usize = 64; const STAT_REGULAR_MODE: u32 = 0o100_644; const STAT_DIRECTORY_MODE: u32 = 0o040_755; const STAT_CHARACTER_MODE: u32 = 0o020_620; +const STAT_FIFO_MODE: u32 = 0o010_600; const DIRECTORY_ENTRY_HEADER_SIZE: usize = 19; const DIRECTORY_ENTRY_ALIGNMENT: usize = 8; const DIRECTORY_TYPE: u8 = 4; @@ -94,6 +96,8 @@ const SIGKILL_BIT: u64 = 1 << (9 - 1); const SIGSTOP_BIT: u64 = 1 << (19 - 1); const CPU_AFFINITY_BYTES: u64 = 8; const INITIAL_CURRENT_DIRECTORY: &[u8] = b"/project"; +const PIPE_CAPACITY_BYTES: usize = 64 * 1024; +const SUPPORTED_PIPE_FLAGS: u64 = OPEN_CLOEXEC | OPEN_NONBLOCK; #[derive(Clone, Copy, Debug, Default)] struct SignalAction { @@ -142,6 +146,19 @@ struct PendingInput { arguments: [u64; 6], } +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +enum PipeEnd { + Read(u64), + Write(u64), +} + +#[derive(Debug, Default)] +struct AnonymousPipe { + bytes: VecDeque, + reader_open: bool, + writer_open: bool, +} + struct DisabledBlockExecutor; impl BlockExecutor for DisabledBlockExecutor { @@ -183,6 +200,11 @@ impl fmt::Display for SyscallEvent { | Syscall::Newfstatat), ) => format_vfs_syscall(formatter, syscall, self.arguments)?, Some(Syscall::Close) => write!(formatter, "close(fd={})", self.arguments[0])?, + Some(Syscall::Pipe2) => write!( + formatter, + "pipe2(pipefd={:#x}, flags={:#x})", + self.arguments[0], self.arguments[1], + )?, Some(Syscall::Fcntl) => write!( formatter, "fcntl(fd={}, command={}, argument={:#x})", @@ -463,6 +485,9 @@ pub struct Process { signal_mask: u64, next_mmap_address: GuestAddress, file_descriptors: BTreeMap, + pipe_descriptors: BTreeMap, + pipes: BTreeMap, + next_pipe_id: u64, descriptor_paths: BTreeMap>, descriptor_flags: BTreeMap, current_directory: Vec, @@ -495,6 +520,9 @@ impl Process { signal_mask: 0, next_mmap_address: GuestAddress::new(MMAP_ARENA_START), file_descriptors: BTreeMap::new(), + pipe_descriptors: BTreeMap::new(), + pipes: BTreeMap::new(), + next_pipe_id: 1, descriptor_paths: BTreeMap::new(), descriptor_flags: BTreeMap::new(), current_directory: INITIAL_CURRENT_DIRECTORY.to_vec(), @@ -743,6 +771,7 @@ impl Process { Some(Syscall::Renameat) => self.dispatch_renameat(filesystem), Some(Syscall::Openat) => self.dispatch_openat(filesystem), Some(Syscall::Close) => self.dispatch_close(filesystem), + Some(Syscall::Pipe2) => self.dispatch_pipe2(), Some(Syscall::Getdents64) => self.dispatch_getdents64(filesystem), Some(Syscall::Lseek) => self.dispatch_lseek(filesystem), Some(Syscall::Readlinkat) => self.dispatch_readlinkat(filesystem), @@ -948,11 +977,7 @@ impl Process { }) .collect::>(); for descriptor in descriptors { - if let Some(handle) = self.file_descriptors.remove(&descriptor) { - let _ = filesystem.close(handle); - } - self.descriptor_paths.remove(&descriptor); - self.descriptor_flags.remove(&descriptor); + let _ = self.close_descriptor(filesystem, descriptor); } } @@ -1061,15 +1086,21 @@ impl Process { self.set_return(Errno::BadFileDescriptor.return_value()); return; }; - let Some(path) = self.descriptor_paths.get(&file_descriptor) else { - self.set_return(Errno::BadFileDescriptor.return_value()); - return; - }; - match filesystem.metadata(path) { - Ok(metadata) => linux_stat_bytes(metadata, path), - Err(error) => { - self.set_return(filesystem_error_return(error)); + if let Some(end) = self.pipe_descriptors.get(&file_descriptor) { + linux_pipe_stat_bytes(match end { + PipeEnd::Read(pipe_id) | PipeEnd::Write(pipe_id) => *pipe_id, + }) + } else { + let Some(path) = self.descriptor_paths.get(&file_descriptor) else { + self.set_return(Errno::BadFileDescriptor.return_value()); return; + }; + match filesystem.metadata(path) { + Ok(metadata) => linux_stat_bytes(metadata, path), + Err(error) => { + self.set_return(filesystem_error_return(error)); + return; + } } } }; @@ -1201,14 +1232,7 @@ impl Process { return Ok(()); } - let Some(handle) = self.file_handle(file_descriptor) else { - self.set_return(Errno::BadFileDescriptor.return_value()); - return Ok(()); - }; - match filesystem.write(handle, &bytes) { - Ok(written) => self.set_return(written as u64), - Err(error) => self.set_return(filesystem_error_return(error)), - } + self.dispatch_descriptor_write(filesystem, file_descriptor, &bytes); Ok(()) } @@ -1432,19 +1456,59 @@ impl Process { self.set_return(Errno::BadFileDescriptor.return_value()); return; }; - let Some(handle) = self.file_descriptors.get(&file_descriptor).copied() else { - self.set_return(Errno::BadFileDescriptor.return_value()); + match self.close_descriptor(filesystem, file_descriptor) { + Ok(()) => self.set_return(0), + Err(error) => self.set_return(error.return_value()), + } + } + + fn dispatch_pipe2(&mut self) { + let pipe_descriptors = GuestAddress::new(self.register(0)); + let flags = self.register(1); + if flags & !SUPPORTED_PIPE_FLAGS != 0 || flags & OPEN_NONBLOCK == 0 { + self.set_return(Errno::InvalidArgument.return_value()); + return; + } + let Some([reader, writer]) = self.allocate_file_descriptors() else { + self.set_return(Errno::TooManyOpenFiles.return_value()); return; }; - match filesystem.close(handle) { - Ok(()) => { - self.file_descriptors.remove(&file_descriptor); - self.descriptor_paths.remove(&file_descriptor); - self.descriptor_flags.remove(&file_descriptor); - self.set_return(0); - } - Err(error) => self.set_return(filesystem_error_return(error)), + let pipe_id = self.next_pipe_id; + let Some(next_pipe_id) = pipe_id.checked_add(1) else { + self.set_return(Errno::TooManyOpenFiles.return_value()); + return; + }; + let mut descriptor_bytes = [0; 8]; + descriptor_bytes[..4].copy_from_slice(&reader.to_le_bytes()); + descriptor_bytes[4..].copy_from_slice(&writer.to_le_bytes()); + if self + .memory + .write(pipe_descriptors, &descriptor_bytes) + .is_err() + { + self.set_return(Errno::Fault.return_value()); + return; } + self.next_pipe_id = next_pipe_id; + self.pipes.insert( + pipe_id, + AnonymousPipe { + bytes: VecDeque::new(), + reader_open: true, + writer_open: true, + }, + ); + self.pipe_descriptors.insert(reader, PipeEnd::Read(pipe_id)); + self.pipe_descriptors + .insert(writer, PipeEnd::Write(pipe_id)); + let descriptor_flags = if flags & OPEN_CLOEXEC == 0 { + 0 + } else { + DESCRIPTOR_CLOEXEC + }; + self.descriptor_flags.insert(reader, descriptor_flags); + self.descriptor_flags.insert(writer, descriptor_flags); + self.set_return(0); } fn dispatch_fcntl(&mut self) { @@ -1453,7 +1517,7 @@ impl Process { return; }; let standard_descriptor = file_descriptor <= 2; - if !standard_descriptor && !self.file_descriptors.contains_key(&file_descriptor) { + if !standard_descriptor && !self.descriptor_exists(file_descriptor) { self.set_return(Errno::BadFileDescriptor.return_value()); return; } @@ -1469,9 +1533,24 @@ impl Process { FCNTL_GET_STATUS_FLAGS => self.set_return(if standard_descriptor { u64::from(file_descriptor != 0) } else { - 0 + match self.pipe_descriptors.get(&file_descriptor) { + Some(PipeEnd::Read(_)) => OPEN_NONBLOCK, + Some(PipeEnd::Write(_)) => OPEN_NONBLOCK | 1, + None => 0, + } }), - FCNTL_SET_STATUS_FLAGS if self.register(2) == 0 => self.set_return(0), + FCNTL_SET_STATUS_FLAGS + if self.pipe_descriptors.contains_key(&file_descriptor) + && self.register(2) == OPEN_NONBLOCK => + { + self.set_return(0); + } + FCNTL_SET_STATUS_FLAGS + if !self.pipe_descriptors.contains_key(&file_descriptor) + && self.register(2) == 0 => + { + self.set_return(0); + } _ => self.set_return(Errno::InvalidArgument.return_value()), } } @@ -1482,7 +1561,7 @@ impl Process { return; }; let standard_descriptor = file_descriptor <= 2; - if !standard_descriptor && !self.file_descriptors.contains_key(&file_descriptor) { + if !standard_descriptor && !self.descriptor_exists(file_descriptor) { self.set_return(Errno::BadFileDescriptor.return_value()); return; } @@ -1501,6 +1580,13 @@ impl Process { } fn dispatch_lseek(&mut self, filesystem: &mut F) { + if u32::try_from(self.register(0)) + .ok() + .is_some_and(|descriptor| self.pipe_descriptors.contains_key(&descriptor)) + { + self.set_return(Errno::IllegalSeek.return_value()); + return; + } let Some(handle) = self.file_handle(self.register(0)) else { self.set_return(Errno::BadFileDescriptor.return_value()); return; @@ -1602,6 +1688,15 @@ impl Process { } fn dispatch_read(&mut self, filesystem: &mut F) { + if let Ok(file_descriptor) = u32::try_from(self.register(0)) + && let Some(end) = self.pipe_descriptors.get(&file_descriptor).copied() + { + match end { + PipeEnd::Read(pipe_id) => self.dispatch_pipe_read(pipe_id), + PipeEnd::Write(_) => self.set_return(Errno::BadFileDescriptor.return_value()), + } + return; + } let Some(handle) = self.file_handle(self.register(0)) else { self.set_return(Errno::BadFileDescriptor.return_value()); return; @@ -1644,6 +1739,50 @@ impl Process { self.set_return(read as u64); } + fn dispatch_pipe_read(&mut self, pipe_id: u64) { + let count = self.register(2); + let Ok(host_count) = usize::try_from(count) else { + self.set_return(Errno::InvalidArgument.return_value()); + return; + }; + if count > self.limits.max_memory_bytes { + self.set_return(Errno::InvalidArgument.return_value()); + return; + } + if host_count == 0 { + self.set_return(0); + return; + } + let Some(pipe) = self.pipes.get(&pipe_id) else { + self.set_return(Errno::BadFileDescriptor.return_value()); + return; + }; + if pipe.bytes.is_empty() { + self.set_return(if pipe.writer_open { + Errno::TryAgain.return_value() + } else { + 0 + }); + return; + } + let read = host_count.min(pipe.bytes.len()); + let bytes = pipe.bytes.iter().take(read).copied().collect::>(); + if self + .memory + .write(GuestAddress::new(self.register(1)), &bytes) + .is_err() + { + self.set_return(Errno::Fault.return_value()); + return; + } + self.pipes + .get_mut(&pipe_id) + .expect("validated pipe remains present") + .bytes + .drain(..read); + self.set_return(read as u64); + } + fn dispatch_pread64(&mut self, filesystem: &mut F) { let Some(handle) = self.file_handle(self.register(0)) else { self.set_return(Errno::BadFileDescriptor.return_value()); @@ -1699,10 +1838,7 @@ impl Process { } fn dispatch_file_write(&mut self, filesystem: &mut F) { - let Some(handle) = self.file_handle(self.register(0)) else { - self.set_return(Errno::BadFileDescriptor.return_value()); - return; - }; + let file_descriptor = self.register(0); let count = self.register(2); let Ok(host_count) = usize::try_from(count) else { self.set_return(Errno::InvalidArgument.return_value()); @@ -1721,12 +1857,57 @@ impl Process { self.set_return(Errno::Fault.return_value()); return; } - match filesystem.write(handle, &bytes) { + self.dispatch_descriptor_write(filesystem, file_descriptor, &bytes); + } + + fn dispatch_descriptor_write( + &mut self, + filesystem: &mut F, + file_descriptor: u64, + bytes: &[u8], + ) { + if let Ok(file_descriptor) = u32::try_from(file_descriptor) + && let Some(end) = self.pipe_descriptors.get(&file_descriptor).copied() + { + match end { + PipeEnd::Read(_) => self.set_return(Errno::BadFileDescriptor.return_value()), + PipeEnd::Write(pipe_id) => self.dispatch_pipe_write(pipe_id, bytes), + } + return; + } + let Some(handle) = self.file_handle(file_descriptor) else { + self.set_return(Errno::BadFileDescriptor.return_value()); + return; + }; + match filesystem.write(handle, bytes) { Ok(written) => self.set_return(written as u64), Err(error) => self.set_return(filesystem_error_return(error)), } } + fn dispatch_pipe_write(&mut self, pipe_id: u64, bytes: &[u8]) { + let Some(pipe) = self.pipes.get_mut(&pipe_id) else { + self.set_return(Errno::BadFileDescriptor.return_value()); + return; + }; + if bytes.is_empty() { + self.set_return(0); + return; + } + if !pipe.reader_open { + self.set_return(Errno::BrokenPipe.return_value()); + return; + } + let available = PIPE_CAPACITY_BYTES.saturating_sub(pipe.bytes.len()); + if available == 0 && !bytes.is_empty() { + self.set_return(Errno::TryAgain.return_value()); + return; + } + let written = available.min(bytes.len()); + pipe.bytes.extend(&bytes[..written]); + self.set_return(written as u64); + } + fn read_guest_path(&self, address: GuestAddress) -> Result, Errno> { let mut path = Vec::new(); for offset in 0..MAX_PATH_BYTES { @@ -1789,14 +1970,63 @@ impl Process { } fn allocate_file_descriptor(&self) -> Option { - let open_count = u32::try_from(self.file_descriptors.len()) + let open_count = u32::try_from(self.file_descriptors.len() + self.pipe_descriptors.len()) .ok()? .checked_add(3)?; if open_count >= self.limits.max_open_files { return None; } (FIRST_FILE_DESCRIPTOR..self.limits.max_open_files) - .find(|descriptor| !self.file_descriptors.contains_key(descriptor)) + .find(|descriptor| !self.descriptor_exists(*descriptor)) + } + + fn allocate_file_descriptors(&self) -> Option<[u32; 2]> { + let mut available = (FIRST_FILE_DESCRIPTOR..self.limits.max_open_files) + .filter(|descriptor| !self.descriptor_exists(*descriptor)); + Some([available.next()?, available.next()?]) + } + + fn descriptor_exists(&self, file_descriptor: u32) -> bool { + self.file_descriptors.contains_key(&file_descriptor) + || self.pipe_descriptors.contains_key(&file_descriptor) + } + + fn close_descriptor( + &mut self, + filesystem: &mut F, + file_descriptor: u32, + ) -> Result<(), Errno> { + if let Some(handle) = self.file_descriptors.get(&file_descriptor).copied() { + filesystem.close(handle).map_err(filesystem_error_errno)?; + self.file_descriptors.remove(&file_descriptor); + self.descriptor_paths.remove(&file_descriptor); + } else if let Some(end) = self.pipe_descriptors.remove(&file_descriptor) { + let pipe_id = match end { + PipeEnd::Read(pipe_id) => { + if let Some(pipe) = self.pipes.get_mut(&pipe_id) { + pipe.reader_open = false; + } + pipe_id + } + PipeEnd::Write(pipe_id) => { + if let Some(pipe) = self.pipes.get_mut(&pipe_id) { + pipe.writer_open = false; + } + pipe_id + } + }; + if self + .pipes + .get(&pipe_id) + .is_some_and(|pipe| !pipe.reader_open && !pipe.writer_open) + { + self.pipes.remove(&pipe_id); + } + } else { + return Err(Errno::BadFileDescriptor); + } + self.descriptor_flags.remove(&file_descriptor); + Ok(()) } fn file_handle(&self, file_descriptor: u64) -> Option { @@ -2180,6 +2410,17 @@ fn linux_terminal_stat_bytes() -> [u8; STAT_SIZE] { bytes } +fn linux_pipe_stat_bytes(pipe_id: u64) -> [u8; STAT_SIZE] { + let mut bytes = [0; STAT_SIZE]; + bytes[..8].copy_from_slice(&1_u64.to_le_bytes()); + bytes[8..16].copy_from_slice(&pipe_id.to_le_bytes()); + bytes[STAT_MODE_OFFSET..STAT_MODE_OFFSET + 4].copy_from_slice(&STAT_FIFO_MODE.to_le_bytes()); + bytes[STAT_LINK_COUNT_OFFSET..STAT_LINK_COUNT_OFFSET + 4].copy_from_slice(&1_u32.to_le_bytes()); + bytes[STAT_BLOCK_SIZE_OFFSET..STAT_BLOCK_SIZE_OFFSET + 4] + .copy_from_slice(&4096_u32.to_le_bytes()); + bytes +} + const fn filesystem_error_return(error: FileSystemError) -> u64 { filesystem_error_errno(error).return_value() } @@ -2213,10 +2454,11 @@ mod tests { use binarrow_runtime_core::{GuestAddress, ResourceLimit, Trap}; use super::{ - AT_FDCWD, DESCRIPTOR_CLOEXEC, ExecutionError, ExecutionEvent, FCNTL_GET_DESCRIPTOR_FLAGS, - IOCTL_CLEAR_CLOSE_ON_EXEC, IOCTL_SET_CLOSE_ON_EXEC, MAIN_THREAD_ID, OPEN_CLOEXEC, - OPEN_DIRECTORY, OPEN_NOCTTY, OPEN_NOFOLLOW, OPEN_PATH, Process, STANDARD_OUTPUT, - STAT_CHARACTER_MODE, STAT_FILE_SIZE_OFFSET, STAT_MODE_OFFSET, STAT_REGULAR_MODE, STAT_SIZE, + AT_FDCWD, AnonymousPipe, DESCRIPTOR_CLOEXEC, ExecutionError, ExecutionEvent, + FCNTL_GET_DESCRIPTOR_FLAGS, IOCTL_CLEAR_CLOSE_ON_EXEC, IOCTL_SET_CLOSE_ON_EXEC, + MAIN_THREAD_ID, OPEN_CLOEXEC, OPEN_DIRECTORY, OPEN_NOCTTY, OPEN_NOFOLLOW, OPEN_NONBLOCK, + OPEN_PATH, PIPE_CAPACITY_BYTES, Process, STANDARD_OUTPUT, STAT_CHARACTER_MODE, + STAT_FIFO_MODE, STAT_FILE_SIZE_OFFSET, STAT_MODE_OFFSET, STAT_REGULAR_MODE, STAT_SIZE, SyscallEvent, SyscallOutcome, }; @@ -3096,6 +3338,122 @@ mod tests { assert_eq!(process.register(0), 0); } + #[test] + fn nonblocking_pipe_round_trips_bytes_and_reports_eof_after_close() { + let image = load_hello(ProcessConfig::default(), 1, MESSAGE_ADDRESS); + let mut process = Process::new(image).unwrap(); + let mut filesystem = NullFileSystem; + let pipe_descriptors = process.state.sp().checked_sub(32).unwrap(); + let source = process.state.sp().checked_sub(64).unwrap(); + let destination = process.state.sp().checked_sub(96).unwrap(); + process.memory.write(source, b"pipe data").unwrap(); + process.state.set_x(0, pipe_descriptors.get()).unwrap(); + process + .state + .set_x(1, OPEN_NONBLOCK | OPEN_CLOEXEC) + .unwrap(); + + process.dispatch_pipe2(); + + assert_eq!(process.register(0), 0); + let mut descriptors = [0; 8]; + process + .memory + .read_exact(pipe_descriptors, &mut descriptors) + .unwrap(); + let reader = u32::from_le_bytes(descriptors[..4].try_into().unwrap()); + let writer = u32::from_le_bytes(descriptors[4..].try_into().unwrap()); + assert_eq!((reader, writer), (3, 4)); + assert_eq!( + process.descriptor_flags.get(&reader), + Some(&DESCRIPTOR_CLOEXEC) + ); + + let stat = process.state.sp().checked_sub(256).unwrap(); + process.state.set_x(0, u64::from(reader)).unwrap(); + process.state.set_x(1, stat.get()).unwrap(); + process.dispatch_fstat(&mut filesystem); + assert_eq!(process.register(0), 0); + let mut stat_bytes = [0; STAT_SIZE]; + process.memory.read_exact(stat, &mut stat_bytes).unwrap(); + assert_eq!( + u32::from_le_bytes( + stat_bytes[STAT_MODE_OFFSET..STAT_MODE_OFFSET + 4] + .try_into() + .unwrap() + ), + STAT_FIFO_MODE + ); + process.state.set_x(0, u64::from(reader)).unwrap(); + process.state.set_x(1, 0).unwrap(); + process.state.set_x(2, 0).unwrap(); + process.dispatch_lseek(&mut filesystem); + assert_eq!(process.register(0), Errno::IllegalSeek.return_value()); + + process.state.set_x(0, u64::from(writer)).unwrap(); + process.state.set_x(1, source.get()).unwrap(); + process.state.set_x(2, 9).unwrap(); + process.dispatch_file_write(&mut filesystem); + assert_eq!(process.register(0), 9); + + process.state.set_x(0, u64::from(reader)).unwrap(); + process.state.set_x(1, destination.get()).unwrap(); + process.state.set_x(2, 16).unwrap(); + process.dispatch_read(&mut filesystem); + assert_eq!(process.register(0), 9); + let mut bytes = [0; 9]; + process.memory.read_exact(destination, &mut bytes).unwrap(); + assert_eq!(&bytes, b"pipe data"); + + process.state.set_x(0, u64::from(reader)).unwrap(); + process.dispatch_read(&mut filesystem); + assert_eq!(process.register(0), Errno::TryAgain.return_value()); + + process.state.set_x(0, u64::from(writer)).unwrap(); + process.dispatch_close(&mut filesystem); + assert_eq!(process.register(0), 0); + process.state.set_x(0, u64::from(reader)).unwrap(); + process.state.set_x(1, destination.get()).unwrap(); + process.state.set_x(2, 16).unwrap(); + process.dispatch_read(&mut filesystem); + assert_eq!(process.register(0), 0); + process.state.set_x(0, u64::from(reader)).unwrap(); + process.dispatch_close(&mut filesystem); + assert!(process.pipes.is_empty()); + + process.state.set_x(0, pipe_descriptors.get()).unwrap(); + process + .state + .set_x(1, OPEN_NONBLOCK | OPEN_CLOEXEC) + .unwrap(); + process.dispatch_pipe2(); + process.close_on_exec_descriptors(&mut filesystem); + assert!(process.pipe_descriptors.is_empty()); + assert!(process.pipes.is_empty()); + } + + #[test] + fn nonblocking_pipe_reports_full_and_broken_writer_boundaries() { + let image = load_hello(ProcessConfig::default(), 1, MESSAGE_ADDRESS); + let mut process = Process::new(image).unwrap(); + process.pipes.insert( + 1, + AnonymousPipe { + bytes: vec![0; PIPE_CAPACITY_BYTES].into(), + reader_open: true, + writer_open: true, + }, + ); + + process.dispatch_pipe_write(1, b"x"); + assert_eq!(process.register(0), Errno::TryAgain.return_value()); + process.pipes.get_mut(&1).unwrap().reader_open = false; + process.dispatch_pipe_write(1, b"x"); + assert_eq!(process.register(0), Errno::BrokenPipe.return_value()); + process.dispatch_pipe_write(1, b""); + assert_eq!(process.register(0), 0); + } + #[test] fn ioctl_tracks_close_on_exec_and_rejects_other_requests_as_not_tty() { let image = load_hello(ProcessConfig::default(), 1, MESSAGE_ADDRESS); diff --git a/docs/architecture.md b/docs/architecture.md index 2978414..c674f20 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -72,7 +72,7 @@ Implements the Phase 2 ephemeral filesystem behind `binarrow-host-api`. It norma ### `binarrow-linux-runtime` -Consumes a loaded process image, owns its architectural execution state and descriptor table, and repeatedly runs the interpreter to structured supervisor-call stops. The dispatcher implements the process calls reached by the static Rust fixture plus `openat`, `close`, `lseek`, regular-file `read`/`write`, and static-ELF `execve`. `*at` calls resolve absolute paths independently of their directory descriptor and relative paths against `AT_FDCWD` or the retained guest path of an open directory descriptor; `O_PATH`, no-follow, no-controlling-terminal, directory, and close-on-exec flags cover toolchain traversal without exposing host paths. Successful process replacement reads a bounded pathname and argument/environment vectors from the old address space, loads the new executable through `HostFileSystem`, rebuilds the process image, closes `O_CLOEXEC` descriptors, and retains the current directory, credentials, signal mask, ordinary descriptors, resource counters, and trace history. Failed replacement leaves the old process image and descriptors intact. File contents remain behind `HostFileSystem`; guest descriptor allocation, guest-memory copying, errno mapping, and open-file limits remain Linux-runtime responsibilities. Unsupported calls return `ENOSYS`, invalid arguments return Linux errno values, and terminal output is bounded before bytes cross the host trait. Successful termination reports the guest exit code plus instruction, syscall, and output counters. Every completed dispatch also appends a project-owned trace event whose arguments are captured before return-register mutation; an explicit syscall budget bounds trace growth. +Consumes a loaded process image, owns its architectural execution state and descriptor table, and repeatedly runs the interpreter to structured supervisor-call stops. The dispatcher implements the process calls reached by the static Rust fixture plus `openat`, `close`, `lseek`, regular-file `read`/`write`, bounded nonblocking `pipe2`, and static-ELF `execve`. Anonymous pipes are process-owned rather than host filesystem objects: each has a fixed 64 KiB queue, reader/writer lifetime, FIFO `fstat` identity, close-on-exec flags, EOF and broken-pipe behavior, and deterministic `EAGAIN` at empty/full nonblocking boundaries. Blocking pipe creation is rejected until the multi-process scheduler can suspend and wake readers and writers correctly. `*at` calls resolve absolute paths independently of their directory descriptor and relative paths against `AT_FDCWD` or the retained guest path of an open directory descriptor; `O_PATH`, no-follow, no-controlling-terminal, directory, and close-on-exec flags cover toolchain traversal without exposing host paths. Successful process replacement reads a bounded pathname and argument/environment vectors from the old address space, loads the new executable through `HostFileSystem`, rebuilds the process image, closes `O_CLOEXEC` descriptors, and retains the current directory, credentials, signal mask, ordinary descriptors, resource counters, and trace history. Failed replacement leaves the old process image and descriptors intact. File contents remain behind `HostFileSystem`; guest descriptor allocation, guest-memory copying, errno mapping, and open-file limits remain Linux-runtime responsibilities. Unsupported calls return `ENOSYS`, invalid arguments return Linux errno values, and terminal output is bounded before bytes cross the host trait. Successful termination reports the guest exit code plus instruction, syscall, and output counters. Every completed dispatch also appends a project-owned trace event whose arguments are captured before return-register mutation; an explicit syscall budget bounds trace growth. ### `binarrow-browser-runtime` @@ -104,7 +104,7 @@ Phase 4 is complete for the initial Tier-1 scope: bounded profiling, scalar basi Phase 5 is complete. A shared native/browser CPython regression independently installs the standard-library snapshot, a multi-file project image, and a checksum-pinned official `packaging` 26.2 wheel image. The CLI's repeatable image-overlay option mirrors the browser install operation, while the browser disables conflicting controls until each asynchronous image request is acknowledged and persists the result in OPFS. The Linux runtime implements the close-on-exec ioctl requests used by CPython's directory-opening path. Project and third-party filesystem imports, retained wheel metadata, deterministic output, guest-path tracebacks, exception text, and exit status are verified in both hosts. Guest-side package resolution, network indexes, and native wheels remain outside this phase; Phase 6 moves to in-browser C compilation. -Phase 6 is in progress. Its first checkpoint adds static `execve` replacement with bounded guest-vector ingestion and Linux-compatible close-on-exec handling. A checked-in launcher, child ELF, and installable filesystem image exercise the same path in native unit tests, the CLI, the browser-runtime native host, and Chromium. This supplies the first compiler-orchestration primitive without pretending to provide concurrent processes, pipes, or a complete toolchain yet. +Phase 6 is in progress. Static `execve` replacement provides bounded guest-vector ingestion and Linux-compatible close-on-exec handling; the packaged Clang/LLD/musl path now compiles browser-edited source, links it, and executes the resulting ELF. Compiler-format stderr records navigate to editor source positions. A bounded `O_NONBLOCK` `pipe2` checkpoint now moves bytes through the same process in native and Chromium hosts. Concurrent child processes, blocking pipe wakeups, cache policy, and interactive-performance work remain. An official static AArch64 Linux Zig 0.16.0 distribution serves as an ignored LLVM/LLD compatibility probe while the final Clang package is selected and pruned. It now completes its version path and advances `zig cc` through a bounded 20-million-instruction startup/compilation run using project-local caches. The trace-derived additions are floating-point width conversion, NEON `rev32`, directory-relative `*at` operations, path-only descriptors, initial guest environment entries, and precise missing-parent errno behavior. The probe bundle and caches remain under `.tmp`; they are evidence and test input, not a shipped replacement for Clang/LLD. diff --git a/guest-tests/pipe-roundtrip/README.md b/guest-tests/pipe-roundtrip/README.md new file mode 100644 index 0000000..6c460dc --- /dev/null +++ b/guest-tests/pipe-roundtrip/README.md @@ -0,0 +1,11 @@ +# Pipe round-trip fixture + +This deterministic AArch64 Linux fixture creates a close-on-exec nonblocking +pipe, writes `pipe hello` through its writer, reads the bytes through its +reader, copies them to standard output, closes both ends, and exits zero. + +Rebuild it from the repository root while keeping the Zig cache local: + +```sh +TMPDIR="$PWD/.tmp" guest-tests/pipe-roundtrip/build.sh +``` diff --git a/guest-tests/pipe-roundtrip/build.sh b/guest-tests/pipe-roundtrip/build.sh new file mode 100755 index 0000000..42ac79b --- /dev/null +++ b/guest-tests/pipe-roundtrip/build.sh @@ -0,0 +1,27 @@ +#!/bin/sh +set -eu + +fixture_directory=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd) +cache_directory=${TMPDIR:-/tmp}/binarrow-pipe-roundtrip-zig-cache +output=$fixture_directory/pipe-roundtrip.aarch64.elf +zig_version=$(zig version) + +if [ "$zig_version" != "0.16.0" ]; then + echo "pipe-roundtrip requires Zig 0.16.0; found $zig_version" >&2 + exit 1 +fi + +env \ + ZIG_LOCAL_CACHE_DIR="$cache_directory/local" \ + ZIG_GLOBAL_CACHE_DIR="$cache_directory/global" \ + zig cc \ + -target aarch64-linux-musl \ + -nostdlib \ + -static \ + -g0 \ + -Wl,-e,_start \ + -Wl,--build-id=none \ + "$fixture_directory/main.S" \ + -o "$output" + +chmod 0644 "$output" diff --git a/guest-tests/pipe-roundtrip/main.S b/guest-tests/pipe-roundtrip/main.S new file mode 100644 index 0000000..5b0c655 --- /dev/null +++ b/guest-tests/pipe-roundtrip/main.S @@ -0,0 +1,63 @@ +.global _start +.type _start, %function + +_start: + sub sp, sp, #32 + mov x0, sp + mov x1, #0x800 + movk x1, #8, lsl #16 + mov x8, #59 + svc #0 + cbnz x0, .Lfailure + + ldr w19, [sp] + ldr w20, [sp, #4] + + mov w0, w20 + adr x1, message + mov x2, #(message_end - message) + mov x8, #64 + svc #0 + cmp x0, #(message_end - message) + b.ne .Lfailure + + mov w0, w19 + add x1, sp, #8 + mov x2, #(message_end - message) + mov x8, #63 + svc #0 + cmp x0, #(message_end - message) + b.ne .Lfailure + + mov x2, x0 + mov x0, #1 + add x1, sp, #8 + mov x8, #64 + svc #0 + cmp x0, #(message_end - message) + b.ne .Lfailure + + mov w0, w20 + mov x8, #57 + svc #0 + cbnz x0, .Lfailure + mov w0, w19 + mov x8, #57 + svc #0 + cbnz x0, .Lfailure + + mov x0, #0 + mov x8, #93 + svc #0 + +.Lfailure: + mov x0, #1 + mov x8, #93 + svc #0 + +.size _start, .-_start + +.section .rodata +message: + .ascii "pipe hello\n" +message_end: diff --git a/guest-tests/pipe-roundtrip/pipe-roundtrip.aarch64.elf b/guest-tests/pipe-roundtrip/pipe-roundtrip.aarch64.elf new file mode 100644 index 0000000000000000000000000000000000000000..8352f7bb0e1b1adbde4ada1c6e89b3c99ca39a42 GIT binary patch literal 1184 zcmb<-^>JfjWMqH=CWh?{Af65*M9={$@qh`+U|?WyV6b3dVQ^qzXJBPuV_;xl0gFN8 z9T>pc8DR7Sh%f_;=791+c7XIjxgZK81gEv3JQxi%7Hl{x0|SE&R37F|7`*{x0s{jB zjE3n8d|HwMaw}9C9|biTqNE_RAXOnFH76&Zi{XDW!^H>83=<&%nsA6=V*K-@w4Im4Tt*(h6>fT_7`H zY@f`$?9?JF1&{pP)TE-+as?kB7X>3jJwrVUi08175YIC)z`_%z1QJ(_3<`|kD1Zqw z@PJv24BBYo5>Rn;_0kLs3_=VD^WpKNSCpTUSdz$~SCU#$!l0L&pPQSSSHhrIT$x*v zn8cu0oKaj-1YtpF29*ki+|=UY#Pn1c9iN()!l07E5MNx9SX2Ub0i^^eJU|$h{9*o4 z00}ZMFmR(;02PLdaxy?tAS~^{!WEW&;r@r}fP^!Y!@$7657lpg5CXIAKrNC1^AH47 z7#>NWc!0Sbp^Aaw2UOt%s36FFpnL_(pCEl8DFp@w24)Zg4R=5dH9`}I8VeWI!l8c` NRKF5jC4_;l9{^iOcA@|P literal 0 HcmV?d00001 diff --git a/web/index.html b/web/index.html index 2f4d380..de96bda 100644 --- a/web/index.html +++ b/web/index.html @@ -33,6 +33,7 @@ + diff --git a/web/src/probe.ts b/web/src/probe.ts index b2c26bd..2f38f46 100644 --- a/web/src/probe.ts +++ b/web/src/probe.ts @@ -22,6 +22,7 @@ export type FixtureName = | "file-roundtrip" | "project-persistence-write" | "project-persistence-read" + | "pipe-roundtrip" | "vfs-lifecycle" | "system-services" | "terminal-input" diff --git a/web/tests/probe.spec.ts b/web/tests/probe.spec.ts index 0ae0fea..9599b72 100644 --- a/web/tests/probe.spec.ts +++ b/web/tests/probe.spec.ts @@ -181,6 +181,24 @@ test("round trips a file through the bounded in-memory filesystem", async ({ ); }); +test("round trips bytes through a bounded nonblocking pipe", async ({ page }) => { + await page.getByLabel("Fixture").selectOption("pipe-roundtrip"); + await page.getByRole("button", { name: "Start" }).click(); + + await expect(page.getByRole("status")).toHaveText("Guest exited"); + await expect(page.getByLabel("Guest terminal output")).toHaveText( + "pipe hello", + ); + await expect(page.locator("#exit-code")).toHaveText("0"); + await expect(page.locator("#syscall-count")).toHaveText("7"); + await expect(page.getByLabel("System call trace")).toContainText( + "pipe2(pipefd=", + ); + await expect(page.getByLabel("System call trace")).toContainText( + "read(fd=3", + ); +}); + test("persists project files in OPFS across a page reload", async ({ page }) => { await page .getByLabel("Fixture")