Ruby Project Idea: Build Your Own Linux Tools

by

Let’s make a project together!

Linux tools like ps, top & netstat are great.

They give you a lot of information about what’s going on with your system.

  • But how do they work?
  • Where do they get all their information from?
  • How can we use this to build our own tools?

In this post we will recreate three popular Linux tools together.

You’re going to get a 2×1 meal, learn some Ruby tricks & get useful Linux knowledge at the same time! 🙂

Finding Status Information

So let’s try answering the question of where all these tools find their info.

The answer is in the proc filesystem!

If you look inside the /proc directory it will look like a bunch of directories & files, just like any other directory on your computer.

These aren’t real files, it’s just a way for the Linux kernel to expose data to users.

It’s very convenient because they can be treated like normal files, which means that you can read them without any special tools.

In the Linux world a lot of things work like this.

If you want to see another example take a look at the /dev directory.

Now that we understand what we are dealing with, let’s take a look at the contents of the /proc directory…

1
10
104
105
11
11015
11469
11474
11552
11655

This is just a small sample, but you can quickly notice a pattern.

What are all those numbers?

Well, it turns out these are PIDs (Process IDs).

Every entry contains info about a specific process.

If you run ps you can see how every process has a PID associated with it:

PID   TTY      TIME     CMD
15952 pts/5    00:00:00 ps
22698 pts/5    00:00:01 bash

From this we can deduce that what ps does is just iterate over the /proc directory & print the info it finds.

Let’s see what is inside one of those numbered directories:

attr
autogroup
auxv
cgroup
clear_refs
cmdline
comm
cpuset
cwd
environ
exe
fd

That’s just a sample to save space, but I encourage you to take a look at the full list.

Here are some interesting entries:

Entry Description
comm Name of the program
cmdline Command used to launch this process
environ Environment variables that this process was started with
status Process status (running, sleeping…) & memory usage
fd Directory that contains file descriptors (open files, sockets…)

Now that we know this we should be able to start writing some tools!

How to List Running Programs

Let’s start by just getting a list of all the directories under /proc.

We can do this using the Dir class.

Example:

Dir.glob("/proc/[0-9]*")

Notice how I used a number range, the reason is that there are other files under /proc that we don’t care about right now, we only want the numbered directories.

Now we can iterate over this list and print two columns, one with the PID & another with the program name.

Example:

pids = Dir.glob("/proc/[0-9]*")

puts "PID\tCMD"
puts "-" * 15

pids.each do |pid|
  cmd = File.read(pid + "/comm")
  pid = pid.scan(/\d+/).first

  puts "#{pid}\t#{cmd}"
end

And this is the output:

PID    CMD
---------------
1     systemd
2     kthreadd
3     ksoftirqd/0
5     kworker/0
7     migration/0
8     rcu_preempt
9     rcu_bh
10    rcu_sched

Hey, it looks like we just made ps! Yeah, it doesn’t support all the fancy options from the original, but we made something work.

Who Is Listening?

Let’s try to replicate netstat now, this is what the output looks like (with -ant as flags).

Active Internet connections (servers and established)

Proto Recv-Q Send-Q Local Address           Foreign Address         State      
tcp        0      0 127.0.0.1:5432          0.0.0.0:*               LISTEN     
tcp        0      0 192.168.1.82:39530      182.14.172.159:22       ESTABLISHED

Where can we find this information? If you said “inside /proc” you’re right! To be more specific you can find it in /proc/net/tcp.

But there is a little problem, this doesn’t look anything like the netstat output!

0: 0100007F:1538 00000000:0000 0A 00000000:00000000 00:00000000 00000000  1001 0 9216
1: 2E58A8C0:9A6A 9FBB0EB9:0016 01 00000000:00000000 00:00000000 00000000  1000 0 258603

What this means is that we need to do some parsing with regular expressions. For now let’s just worry about the local address & the status.

Here is the regex I came up with:

LINE_REGEX = /\s+\d+: (?<local_addr>\w+):(?<local_port>\w+) \w+:\w+ (?<status>\w+)/

This will give us some hexadecimal values that we need to convert into decimal. Let’s create a class that will do this for us.

class TCPInfo
  def initialize(line)
    @data = parse(line)
  end

  def parse(line)
    line.match(LINE_REGEX)
  end

  def local_port
    @data["local_port"].to_i(16)
  end

  # Convert hex to regular IP notation
  def local_addr
    decimal_to_ip(@data["local_addr"].to_i(16))
  end

  STATUSES = {
    "0A" => "LISTENING",
    "01" => "ESTABLISHED",
    "06" => "TIME_WAIT",
    "08" => "CLOSE_WAIT"
  }

  def status
    code = @data["status"]

    STATUSES.fetch(code, "UNKNOWN")
  end

  # Don't worry too much about this. It's some binary math.
  def decimal_to_ip(decimal)
    ip = []

    ip << (decimal >> 24 & 0xFF)
    ip << (decimal >> 16 & 0xFF)
    ip << (decimal >> 8 & 0xFF)
    ip << (decimal & 0xFF)

    ip.join(".")
  end
end

The only thing left is to print the results in a pretty table format.

require 'table_print'

tp connections

Example output:

STATUS      | LOCAL_PORT | LOCAL_ADDR   
------------|------------|--------------
LISTENING   | 5432       | 127.0.0.1    
ESTABLISHED | 39530      | 192.168.88.46

Yes, this gem is awesome!

I just found about it & looks like I won’t have to fumble around with ljust / rjust again 🙂

Stop Using My Port!

Have you ever seen this message?

Address already in use - bind(2) for "localhost" port 5000

Umm…

I wonder what program is using that port.

Let’s find out:

fuser -n tcp -v 5000

PORT       USER        PID   ACCESS CMD
5000/tcp   rubyguides  30893 F....  nc

Ah, so there is our culprit!

Now we can stop this program if we don’t want it to be running & that will free our port. How did the “fuser” program find out who was using this port?

You guessed it!

The /proc filesystem again.

In fact, it combines two things we have covered already: walking through the process list & reading active connections from /proc/net/tcp.

We just need one extra step:

Find a way to match the open port info with the PID.

If we look at the TCP data that we can get from /proc/net/tcp, the PID is not there. But we can use the inode number.

“An inode is a data structure used to represent a filesystem object.” – Wikipedia

How can we use the inode to find the matching process? If we look under the fd directory of a process that we know has an open port, we will find a line like this:

/proc/3295/fd/5 -> socket:[12345]

The number between brackets is the inode number. So now all we have to do is iterate over all the files & we will find the matching process.

Here is one way to do that:

x =
Dir.glob("/proc/[0-9]*/fd/*").find do |fd|
  File.readlink(fd).include? "socket:[#{socket_inode}]" rescue nil
end

pid  = x.scan(/\d+/).first
name = File.readlink("/proc/#{pid}/exe")

puts "Port #{hex_port.to_i(16)} in use by #{name} (#{pid})"

Example output:

Port 5432 in use by /usr/bin/postgres (474)

Please note that you will need to run this code as root or as the process owner.

Otherwise you won’t be able to read the process details inside /proc.

Conclusion

In this post you learned that Linux exposes a lot of data via the virtual /proc filesystem. You also learned how to recreate popular Linux tools like ps, netstat & fuser by using the data under /proc.

Don’t forget to subscribe to the newsletter below so you don’t miss the next post. 🙂

11 thoughts on “Ruby Project Idea: Build Your Own Linux Tools”

Comments are closed.