Strace in depth (profiling system calls)

Hi,
Software developers sometime find it necessary to delve into binary files and get a better grasp of what exactly is done under the hood while running an executable, Does it affect the overall system performance?
Which type of system calls are getting invoked?


I spoke briefly few years ago about strace Link.
With strace you can  obtain a lot of information on kernel  calls while your program is still executing, allowing you to follow the flow of the process “live”, and save the strace output, in order to comfortably analyze it afterwards “offline”.

we can easily use strace for this task, but sometimes the huge amount of output might be too cluttered so here are few more tips:

tip #1: strace the output into a file and use the verbose option

               strace -v -o dump_file.txt   bin_file

              The verbose flag would assure you get all arguments per system call
              invocation.

              This way you can get some answers for your questions:

               1) What system calls are employed by application?
               2)  Which files does application touch?
               3) What arguments are being passed to each system call?
               4) Which system calls are failing, and why? (errono)

tip #2: strace by process-id and get time spent on each system call

               strace -v -o dump_file.txt   bin_file
               or
               strace -p <pid> -o dump_file.txt

              The -T Shows the time spent on each system call.  This records the time
              difference between the beginning and the end of each  system call.

tip #3: Apply filters on the system calls

               there are about 400 system-calls, and sometimes we would like to avoid
               getting irrelevant info system-calls. for example we would like
               to investigate only 2 system calls open() close(), so we will use the -e flag:

                    

               strace -e trace=open,close -o dump_file.txt  bin_file

               we can use the groups criteria for filtering, there are 7 categories:

               file          -  Trace all system calls which take a file name as an argument

               process  -  Trace all system calls which involve process management.

                                   like fork, wait, and exec steps of a process.

               network - Trace all the network related system calls.

               signal     - Trace all signal related system calls.

               ipc           - Trace all IPC related system calls.

               memory  -  Trace all memory mapping related system calls.

               desc         - Trace all file descriptor related system calls. 

               for example: Getting all system calls regarding network operations:

               strace -e trace=network -o dump_file.txt  bin_file

tip #4: Getting a much more clear picture of the how the system-calls are
              distributed.

              strace -c -w -S time bin_file

              The -c Count time, calls, and errors for each system call

              The -w Summarise the time difference between the beginning 
              and end of each system call

              The -S Sort the output of the histogram printed by the -c option 
              by the specified criterion. Legal values are time,calls, name, 
              and nothing.

              for example: lets check the system calls distribution when 
                                    invoking dd the command:

              I have generated a simple barplot (with R) to present it more 
              visually:

For summary with strace you can easily analyze and investigate malicious code,
very useful for daily usage.
I hope you enjoyed the post, let me know if you would like me to cover other topics. till next time, bye!