pprof #

Reads and writes pprof profiles, the sampled stack format that profile.proto defines and that go tool pprof, Perfetto, speedscope and pyroscope all open.

Overview #

Most profile viewers read pprof, so a profiler that writes it gets those viewers for free. A pprof profile is a list of samples. Each sample holds a stack of location ids, leaf first, and one value for each sample type the profile declares, such as alloc_objects/count, alloc_space/bytes or cpu/nanoseconds. A location points to lines, a line to a function, and a location may also point to the mapping its code was loaded from.

The wire format stores each string once, in a table, and refers to it by index. The encoder builds that table and the decoder resolves it, so Pprof.Profile.t holds ordinary OCaml strings. Ids are kept as ids, as in profile.proto. The decoder also checks the cross-references the specification requires, so a profile it returns as Ok is one that external tools accept.

A profile file is protobuf wrapped in gzip. Pprof.to_string compresses unless told otherwise. Pprof.of_string looks for the gzip magic bytes and accepts a bare protobuf message when they are absent, which is how profiles often arrive in HTTP bodies and archives.

Requires OCaml >= 4.14.

Install #

$ opam install pprof

Usage #

This builds an allocation profile with one sample and encodes it. The output starts with the gzip magic:

let fn : Pprof.Function.t =
  {
    id = 1L;
    name = "List.map";
    system_name = "camlStdlib__List__map_267";
    filename = "list.ml";
    start_line = 96L;
  }

let loc : Pprof.Location.t =
  {
    id = 1L;
    mapping_id = 0L;
    address = 0L;
    lines = [ { function_id = 1L; line = 100L; column = 0L } ];
    is_folded = false;
  }

let heap : Pprof.t =
  {
    Pprof.Profile.empty with
    sample_types =
      [
        { typ = "alloc_objects"; unit = "count" };
        { typ = "alloc_space"; unit = "bytes" };
      ];
    samples = [ { location_ids = [ 1L ]; values = [ 3L; 96L ]; labels = [] } ];
    locations = [ loc ];
    functions = [ fn ];
    period_type = Some { typ = "space"; unit = "bytes" };
    period = 524_288L;
  }

# String.sub (Pprof.to_string heap) 0 2 = "\x1f\x8b"
- : bool = true

Decoding the result gives the profile back:

# match Pprof.of_string (Pprof.to_string heap) with
  | Ok p -> Fmt.str "%a" Pprof.Profile.pp p
  | Error e -> Pprof.Error.to_string e
- : string =
"[alloc_objects/count, alloc_space/bytes] period=524288 1 samples, 1 locations, 1 functions, 0 mappings"

Pprof.Profile.validate refuses a sample that names a location the profile does not declare:

# Pprof.Profile.validate
    { heap with
      samples = [ { location_ids = [ 9L ]; values = [ 1L; 2L ]; labels = [] } ] }
  |> Result.map_error Pprof.Error.to_string
- : (unit, string) result =
Error "sample.location_id references location id 9, which is not declared"

API #

The Pprof module reads with of_string, of_string_exn, of_reader and of_reader_exn, and writes with to_string and to_writer. The writers take ?compress (default true) to choose whether to gzip. The readers take ?max_size, a bound on the decompressed bytes (256 MiB by default), because a small gzip member can expand to any size.

Each profile.proto message has its own module, a record with equal and pp:

• Pprof.Profile is the whole document, with empty and validate.
• Pprof.Sample is a stack with its values and labels.
• Pprof.Value_type says what a value measures and in which unit.
• Pprof.Label is a string or numeric annotation on a sample.
• Pprof.Location, Pprof.Line, Pprof.Function and Pprof.Mapping are the symbolisation tables.

Pprof.Error adds the pprof failures to Loc.Error.kind. A caller can match on constructors such as Missing_string_table or Dangling_id { table; id; from } without parsing an error message.

Related work #

Google defines profile.proto, and its Go implementation lives at github.com/google/pprof. This library follows the same split between wire indices and resolved strings as that code's preEncode and postDecode, and its validation rules are those of CheckValid.

OCaml profilers each have their own format. memtrace writes CTF traces, while landmarks and spacetime write their own dumps. nox-catapult writes Chrome trace events, which record durations on a timeline instead of sampled stacks, and ocaml-observe has a Fuchsia trace writer. A sampler that exports pprof through this package can be read by the common viewers; obs pprof uses it to convert memtrace traces.

Protobuf encoding comes from nox-protobuf and gzip from bytesrw.zlib.

License #

ISC. See LICENSE.md.