Re: [RFC] Adding Support for Firefox's Gecko Profile Format

[Anup Sharma]

On Wed, Aug 16, 2023 at 10:40:07PM +0530, Anup Sharma wrote:
> On Thu, Aug 10, 2023 at 01:31:09AM +0530, Anup Sharma wrote:
> > On Thu, Aug 03, 2023 at 12:29:51AM +0530, Anup Sharma wrote:
> > > On Wed, Jul 26, 2023 at 04:22:51PM -0300, Arnaldo Carvalho de Melo wrote:
> > > > Em Wed, Jul 26, 2023 at 11:34:17PM +0530, Anup Sharma escreveu:
> > > > > Dear all,
> > > > > 
> > > > > I want to share a recent development in our project that
> > > > > I have been working on this week. I have successfully
> > > > > implemented a new feature that allows users to directly
> > > > > open the browser and visualize the output of perf.data
> > > > > on Firefox Profiler.
> > > > 
> > > > Excellent news!
> > > >  
> > > > > To enhance user experience, I have made sure this option
> > > > > is enabled by default. Additionally, I've also added a
> > > > > handy command line argument, "--save-only," which allows
> > > > > users to save the output file as needed. This enhancement
> > > > > replaces the previous standard output method, providing a
> > > > > much-improved experience.
> > > > > 
> > > > > Moreover, Everything is processed locally, and no data is
> > > > > sent to any server.
> > > > 
> > > > That addresses the worry about sending possibly sensitive data over the
> > > > network by default, great.
> > > >  
> > > > > During this implementation, I had the opportunity to delve
> > > > > into several exciting topics, including threading, HTTP server,
> > > > > and CORS mechanisms. The learning process has been both
> > > > > challenging and rewarding.
> > > > 
> > > > Cool that you're so excited learning all this.
> > > >  
> > > > > Currently, I am in the process of performing some clean-up tasks.
> > > > > Once that is complete, I will be sharing the next version of the
> > > > > series, which includes this fantastic new feature, with all of
> > > > > you very soon.
> > > > > 
> > > > > Thank you for your ongoing support and encouragement.
> > > > 
> > > > Keep it up!
> > > 
> > > This week, I'm working on documenting the gecko converter for our wiki.
> > > I've dedicated a significant amount of time to think about the content
> > > and how to approach it. So far, I've written sections covering the
> > > introduction to Firefox Profiler and the formatting guidelines(not
> > > published yet).
> > > 
> > > However, I've left the usage part blank because I'm uncertain about how
> > > to enable gecko.py arguments when executing the "perf script gecko" command.
> > > I've looked at examples like flamegraph-report and sctop-report, but I'm
> > > still confused. I attempted to enable the script to take the gecko.py
> > > command example "perf script report gecko --save-only=profiler.json,"
> > > but this caused issues when running "perf script gecko -a sleep 3" again.
> > > This has left me feeling rather puzzled, and I would appreciate discussing
> > > this during our office hour, or any help here.
> > > 
> > > Due to this blocker, I haven't been able to send the updated patch
> > > for test_gecko yet. However, I hope I'll find a solution soon and overcome
> > > this obstacle.
> > 
> > Hello All,
> > As discussed during last office hour, I was little occupied this week, so
> > I haven't been able to make much progress on my TODO list. Nonetheless,
> > I've resent a server patch and test script patch for your review. If I can
> > get it reviewed it would be great to make further progress.
> > 
> > I'm also working on writing documentation for our wiki. I've already written a
> > bit, and you can see it here: https://perf.wiki.kernel.org/index.php/Tutorial#Firefox_Profiler.
> > If you have any thoughts or suggestions, I'd appreciate hearing them. I'll add
> > more details once I've sorted out some issues I'm having with certain commands.
> > 
> > This week, I do not have a specific agenda. Hence, if we opt to skip this week's
> > office hour, it wouldn't pose any inconvenience. Nevertheless, I am available if
> > there are matters you wish to discuss.
> > 
> > Here's what I have in mind to work on next:
> > 
> > 1. Making sure that gecko script args work with commands, like
> > (perf script gecko --save-only=profile.json) or (perf script gecko --kernel-color=green),which
> > aren't working right now.
> 
> I have done this part and sent the patch for review.
> https://lore.kernel.org/all/ZNf7W+EIrrCSHZN0@yoga/
> 
> > 2. Trying out "perf record -p firefox" or using it with a specific process ID to
> > see if the current script supports these recording commands.
> 
> This is working with the existing script. I've tested it and documented it as well.
> https://perf.wiki.kernel.org/index.php/Tutorial#Firefox_Profiler
> 
> > 3. Adding page fault and one more event in stackTable in gecko script.
> 
> I have few doubt regarding this will discuss in office hour.
> 
> > 4. Making the documentation better.
> 
> I am trying to make it better, but I'm not sure if it's good enough. Here's the link:
> https://perf.wiki.kernel.org/index.php/Tutorial#Firefox_Profiler
> 
> I also need to understand on how to prepare the final report for GSoC and
> planning to record a 5 min tutorial video on how to use the script.
> 
> Moreover, I need to make the script run as a root user.
> 
> This are few TODOs I have in mind. Please let me know if you have any suggestions.

I'm thrilled to share that I've received an offer from Nutanix Technologies for the
MTS-1 Kernel Development role, and I couldn't be more excited about it. I want to
express my gratitude for all your support. I'm currently on vacation, but I'll be
back in a few days to continue making contributions.

> > Thanks,
> > Anup
> > 
> > > > - Arnaldo
> > > > 
> > > > > Thanks,
> > > > > Anup
> > > > > 
> > > > > > > Regards,
> > > > > > > Anup
> > > > > > > > > 
> > > > > > > > >>
> > > > > > > > >> The second challenge revolved around the usage of event hooks provided with the perf script
> > > > > > > > >> python event handlers. I found myself deliberating between two approaches. The first
> > > > > > > > >> approach involved creating custom functions that would be called using the event
> > > > > > > > >> handlers. These functions would then save the data in an organized format within
> > > > > > > > >> globally created data structures. The alternative approach was to write the entire
> > > > > > > > >> logic inside the event handlers themselves.
> > > > > > > > >>
> > > > > > > > >> Additionally, I contemplated whether it would be more suitable to handle the creation of
> > > > > > > > >> a Gecko format for JSON and the profile format within the same script or to separate
> > > > > > > > >> them into different scripts.
> > > > > > > > >>
> > > > > > > > >> I will discuss this points during tomorrow's office hour.
> > > > > > > > >>
> > > > > > > > >> However, I have gained a deeper understanding of the problem at hand and will use this
> > > > > > > > >> knowledge to make more informed decisions and progress more effectively in the coming weeks.
> > > > > > > > > 
> > > > > > > > > Sounds like you did a good amount of research on how perf script
> > > > > > > > > works.  We can talk about the details in the meeting.
> > > > > > > > > 
> > > > > > > > > Thanks,
> > > > > > > > > Namhyung
> > > > > > > > 
> > > > 
> > > > -- 
> > > > 
> > > > - Arnaldo