Show allocation data in the cli

[fatadel]

Main | Deploy preview

The cli only ever summarized sample timing, so JS and native allocation data was invisible.

Add --strategy to the samples, functions and annotate commands to pick a data source per invocation, and a "strategy" command to set it for the session. Allocation sources weigh the call tree in bytes, and "thread info" lists the sources each thread has.

Closes #6183

---

Profiles to explore:

Normal · JS Allocations · Native Allocations

Example usages:

1. pq load https://share.firefox.dev/4w9ddss
2. pq thread info  # see the available data sources
3. pq thread samples --strategy native-deallocations-sites

# "Sticky" change
1. pq strategy native-deallocations-sites
2. pq status  # See the data source

[codecov]

Codecov Report

❌ Patch coverage is 58.69565% with 38 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.78%. Comparing base (7abc8b8) to head (8ccddd1).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
src/profile-query/index.ts	41.17%	20 Missing ⚠️
src/profile-query/call-tree-strategy.ts	79.31%	6 Missing ⚠️
src/profile-query/formatters/thread-info.ts	53.84%	6 Missing ⚠️
src/profile-query/formatters/call-tree.ts	50.00%	5 Missing ⚠️
src/profile-query/function-annotate.ts	83.33%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6246      +/-   ##
==========================================
- Coverage   83.82%   83.78%   -0.04%     
==========================================
  Files         350      351       +1     
  Lines       37744    37801      +57     
  Branches    10646    10676      +30     
==========================================
+ Hits        31638    31672      +34     
- Misses       5677     5700      +23     
  Partials      429      429              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[fatadel]

I've now rebased this on the latest main and resolved the conflicts. @canova @mstange kind reminder to review this 😌

[canova]

Thanks! I'm reviewing the code still but while doing some manual testing I found this issue, sending earlier:

I executed these:

pq load https://share.firefox.dev/4w9ddss # the example in the PR
pq thread samples --strategy native-deallocations-memory

And I see these:

Top Functions (by total bytes):
  (For a call tree starting from these functions, use: profiler-cli thread samples-top-down)

  f-329. XUL!mozilla::dom::Document::FlushPendingNotifications(mozilla::ChangesToFlush) - total: -16B (-0.0%)
  f-2091. XUL!WrapInWrapList(mozilla::nsDisplayListBuilder*, nsIFrame*, mozilla::nsDisplayList*, mozilla::) - total: -16B (-0.0%)
  f-2092. XUL!mozilla::nsDisplayList::Clear() - total: -16B (-0.0%)
  f-2093. XUL!mozilla::nsDisplayList::DeallocateNodes() - total: -16B (-0.0%)
  f-2096. XUL!mozilla::nsDisplayList::Deallocate(mozilla::LinkedListNode<mozilla::nsDisplayItem*>*) - total: -16B (-0.0%)
  f-2351. XUL!mozilla::nsDisplayWrapList::nsDisplayWrapList(mozilla::nsDisplayListBuilder*, nsIFrame*, mozilla::) - total: -16B (-0.0%)
  f-7285. XUL!mozilla::nsDisplayList::AppendNewToTop<mozilla::nsDisplayAsyncZoom, mozilla::>(mozilla::) - total: -16B (-0.0%)
  f-7286. XUL!mozilla::nsDisplayList::AppendNewToTopWithIndex<mozilla::nsDisplayAsyncZoom, mozilla::>(mozilla::) - total: -16B (-0.0%)
  f-7287. XUL!mozilla::MakeDisplayItemWithIndex<mozilla::nsDisplayAsyncZoom, mozilla::>(mozilla::) - total: -16B (-0.0%)
  f-7288. XUL!mozilla::nsDisplayAsyncZoom::nsDisplayAsyncZoom(mozilla::nsDisplayListBuilder*, nsIFrame*, mozilla::) - total: -16B (-0.0%)
  f-7289. XUL!mozilla::nsDisplayOwnLayer::nsDisplayOwnLayer(mozilla::nsDisplayListBuilder*, nsIFrame*, mozilla::) - total: -16B (-0.0%)
  f-7526. XUL!mozilla::dom::Document::FlushPendingNotifications(mozilla::FlushType) - total: -16B (-0.0%)
  f-7529. XUL!mozilla::image::VectorImage::CreateSurface(mozilla::image::SVGDrawingParameters const&, gfxDrawable*, bool&) - total: -16B (-0.0%)
  f-7530. XUL!mozilla::image::SVGDocumentWrapper::FlushImageTransformInvalidation() - total: -16B (-0.0%)
  f-7531. XUL!mozilla::image::SVGDocumentWrapper::FlushLayout() - total: -16B (-0.0%)
  f-7794. XUL!std::__1::vector<RefPtr<mozilla::>>::push_back[abi:sqn210106](RefPtr<mozilla::gfx::DrawEventRecorderPrivate> const&) - total: -16B (-0.0%)
  f-7795. XUL!std::__1::vector<RefPtr<>, std::>emplace_back<RefPtr<mozilla::gfx::DrawEventRecorderPrivate> const&>(RefPtr<>) - total: -16B (-0.0%)
  f-7796. XUL!std::__1::vector<RefPtr<>, std::>__emplace_back_slow_path<RefPtr<mozilla::gfx::DrawEventRecorderPrivate> const&> - total: -16B (-0.0%)
  f-7801. XUL!std::__1::__split_buffer<RefPtr>::__split_buffer(unsigned long, unsigned long, std::__1::allocator<RefPtr<> >&) - total: -16B (-0.0%)
  f-7802. XUL!std::__1::__allocate_at_least[abi:sqn210106]<std::__1::allocator<RefPtr<mozilla::gfx::> > > - total: -16B (-0.0%)
  f-7803. XUL!std::__1::allocator<RefPtr<mozilla::gfx::DrawEventRecorderPrivate> >::allocate[abi:sqn210106](unsigned long) - total: -16B (-0.0%)
  f-7805. XUL!std::__1::__libcpp_allocate[abi:sqn210106]<RefPtr<mozilla::gfx::DrawEventRecorderPrivate> > - total: -16B (-0.0%)
  f-8020. XUL!nsTArray<RefPtr<nsRefreshDriver> >::Clone() const - total: -16B (-0.0%)
  f-8021. XUL!nsTArray_Impl<RefPtr<>>::Assign<nsTArrayInfallibleAllocator, nsTArrayInfallibleAllocator>(nsTArray_Impl) - total: -16B (-0.0%)
  f-8022. XUL!nsTArray_Impl<RefPtr<>>::AssignInternal<nsTArrayInfallibleAllocator, RefPtr<nsRefreshDriver> >(RefPtr<>) - total: -16B (-0.0%)
  f-8059. XUL!mozilla::layers::SetPaintRequestTime(nsIContent*, mozilla::TimeStamp const&) - total: -16B (-0.0%)
  f-8205. XUL!mozilla::Vector<js::jit::PerfSpewer::DebugEntry, (unsigned long)0, js::>::convertToHeapStorage(unsigned long) - total: -16B (-0.0%)
  f-9157. XUL!nsTArray<std::__1::unique_ptr>::AppendElement<std::__1::unique_ptr<mozilla::net::ChannelEvent, std::__1::> >() - total: -16B (-0.0%)
  f-9158. XUL!nsTArray_Impl<std::__1::>::AppendElementInternal<nsTArrayInfallibleAllocator, std::__1::unique_ptr<> >(std::) - total: -16B (-0.0%)
  f-594. CoreFoundation!CFDictionaryCreateMutable - total: -32B (-0.0%)
  f-596. CoreFoundation!__NSDictionaryM_new - total: -32B (-0.0%)
  f-598. libsystem_malloc.dylib!_malloc_type_calloc_outlined - total: -32B (-0.0%)
  f-599. libsystem_malloc.dylib!_malloc_zone_calloc_instrumented_or_legacy - total: -32B (-0.0%)
  f-857. CoreFoundation!__CFAllocateObject - total: -32B (-0.0%)
  f-917. libobjc.A.dylib!class_createInstance - total: -32B (-0.0%)
  f-1576. Task UITimerCallback_timer - total: -32B (-0.0%)
  f-2097. XUL!mozilla::nsDisplayListBuilder::Destroy(mozilla::DisplayListArenaObjectId, void*) - total: -32B (-0.0%)
  f-2098. XUL!nsPresArena<(unsigned long)32768, mozilla::DisplayListArenaObjectId, >Free(mozilla::DisplayListArenaObjectId, void*) - total: -32B (-0.0%)
  f-2099. XUL!nsTArray<void*>::AppendElement<void*&>(void*&) - total: -32B (-0.0%)
  f-2101. XUL!nsTArray_Impl<void*, nsTArrayInfallibleAllocator>AppendElementInternal<nsTArrayInfallibleAllocator, void*&>(void*&) - total: -32B (-0.0%)
  f-3051. XUL!mozilla::UITimerCallback::Notify(nsITimer*) - total: -32B (-0.0%)
  f-4854. XUL!mozilla::net::nsStandardURL::NormalizeIDN(nsTSubstring<char> const&, nsTSubstring<char>&) - total: -32B (-0.0%)
  f-4855. XUL!NS_DomainToDisplayAndASCII(nsTSubstring<char> const&, nsTSubstring<char>&, nsTSubstring<char>&) - total: -32B (-0.0%)
  f-4856. XUL!mozilla_net_domain_to_display_and_ascii_impl - total: -32B (-0.0%)
  f-4857. XUL!nsstring::nsACString::assign - total: -32B (-0.0%)
  f-4858. XUL!Gecko_AssignCString - total: -32B (-0.0%)
  f-5656. XUL!do_GetWeakReference(nsISupports*, nsresult*) - total: -32B (-0.0%)
  f-5657. XUL!NS_GetWeakReference(nsISupports*, nsresult*) - total: -32B (-0.0%)
  f-5658. XUL!nsCOMPtr<nsISupportsWeakReference>::nsCOMPtr<nsISupports>() - total: -32B (-0.0%)
  f-5660. XUL!nsCOMPtr<nsISupportsWeakReference>assign_from_qi_with_error<nsISupports>(nsQueryInterfaceWithError<nsISupports>) - total: -32B (-0.0%)

They all show ~-0.0% which doesn't seem right. I think that's because we don't look at the absolute values, and that's why the negative values are closer to zero looks at the top. We have a logic in the web side that look at absolute values for exatly this. Tbh, I don't know why this is different for the cli yet, because ideally we should share the same code for the web and the cli for this.

[canova]

Also I have a question:
Is pq thread strategy native-deallocations-sites changing the state for that thread only or for every thread (like persistent across threads)? If we are changing for the whole profile we shouldn't put it as a subcommand under thread. It would be good to do something similar to pq filter, as a command and not subcommand.

[canova]

Also got another issue while testing:

$ pq load https://share.firefox.dev/4w9ddss
$ pq thread samples-bottom-up --strategy native-deallocations-memory
Error: Cannot read properties of null (reading 'toFixed')

[fatadel]

Thanks a lot for the thorough review, @canova! I've now addressed the issues.