RubyGuides
Share this post!

The Ultimate Guide to HTTP Requests in Ruby

If you’d like to get information from a website, or if you’d like to submit forms, upload files…

…you’ll need to send an HTTP request & then process the response.

In this article you’ll learn how to:

  • Make a simple HTTP request using net/http
  • Send SSL requests
  • Submit data using a POST request
  • Send custom headers
  • Choose the best HTTP client for your situation

Let’s do this!

How to Send an HTTP Request

Ruby comes with a built-in http client, it’s called net/http & you can use it to send any kind of request you need.

Here’s a net/http example:

require 'net/http'

Net::HTTP.get('example.com', '/index.html')

This will return a string with the HTML content of the page.

But often you want more than the HTML content.

Like the HTTP response status.

Without the response status you don’t know if your request was successful, or if it failed.

This is how you get that:

response = Net::HTTP.get_response('example.com', '/')

response.code
# 200

Now if you want the response content you call the body method:

response.body

How to Use the HTTParty Gem

There are many gems that can make things easier for you.

One of these gems is httparty.

Here’s how to use it:

require 'httparty'

response = HTTParty.get('http://example.com')

response.code
# 200

response.body
# ...

The benefits of using an HTTP gem:

  • It’s easier to use.
  • There is no separate get_response method, get already gives you a response object.
  • As you’ll see in the next section they make SSL request transparent

Sending SSL Requests

If you try to send an SSL request with net/http:

Net::HTTP.get_response("example.com", "/", 443)

You get:

Errno::ECONNRESET: Connection reset by peer

You’d have to do this instead:

net = Net::HTTP.new("example.com", 443)

net.use_ssl = true
net.get_response("/")

Save yourself some work & use a gem 🙂

How to Submit Data With a Post Request

A GET request is used to request information.

Like downloading an image, css, javascript…

But if you want to submit information use a POST request.

Here’s an example:

HTTParty.post("http://example.com/login", body: { user: "test@example.com", password: "chunky_bacon" })

To upload a file you’ll need a multipart request, which is not supported by HTTParty.

You can use the rest client gem:

require 'rest-client'

RestClient.post '/profile', file: File.new('photo.jpg', 'rb')

Or the Faraday gem:

require 'faraday'

conn =
Faraday.new do |f|
  f.request :multipart
  f.request :url_encoded

  f.adapter :net_http
end

file_io = Faraday::UploadIO.new('photo.jpg', 'image/jpeg')

conn.post('http://example.com/profile', file: file_io)

How to Send Custom HTTP Headers

You can send custom headers with an HTTP request.

This helps you send extra data with your request, including cookies, user-agent, and caching information.

Here’s how:

Faraday.new('http://example.com', headers: { 'User-Agent' => 'test' }).get

I’m creating a Faraday object (using new), then calling get on it.

This doesn’t work if you call get directly.

Choosing The Best Ruby HTTP Client

There are many HTTP clients available in Ruby.

But which one should you choose?

To help you make this decision I prepared a comparison table for you.

Here it is:

REPO STARS RECENT_COMMITS LAST_COMMIT LATEST_RELEASE CREATED_DATE
lostisland/faraday 4103 18 2018-05-28 21:42:12 2018-05-23 15:15:54 2009-12-10 17:14:55
rest-client/rest-client 4441 0 2017-09-20 04:00:46 2017-07-05 08:43:12 2009-12-07 19:34:29
typhoeus/typhoeus 3332 9 2018-07-30 21:21:26 2017-08-21 09:47:17 2009-02-18 23:14:50
jnunemaker/httparty 4596 17 2018-03-30 13:54:32 2018-03-30 13:54:32 2008-07-28 20:55:11
excon/excon 894 38 2018-08-02 16:01:07 2018-03-27 15:39:37 2009-10-25 17:50:46
httprb/http 2225 27 2018-06-13 16:38:36 2018-04-25 13:49:26 2011-10-06 04:19:10

 
Looking at popularity & how well maintained these gems are is a good first step into evaluating all the options.

But what are the technical differences between them?

Many of these gems are wrappers around the net/http library:

  • excon
  • httparty
  • rest client

While Faraday is an adapter gem, it can use different implementations.

And Typhoeus is a wrapper around the C libcurl library.

Typhoeus For Multi-Threading

Typhoeus supports concurrent requests without having to write any concurrent code. This makes Typhoeus the best choice if you are looking to maximize your requests per second.

To use Typhoeus with multi-threading:

require 'typhoeus'

hydra   = Typhoeus::Hydra.hydra
request = Typhoeus::Request.new("https://www.rubyguides.com")

hydra.queue(request)
hydra.queue(request)

hydra.run

Faraday Gem For Adaptability

Faraday allows you to choose any implementation from net/http, to Typhoeus, or even rest-client all from the same interface.

It defaults to net/http, but you can change it like this:

Faraday.default_adapter = :typhoeus

Another interesting thing about Faraday is that it supports middleware.

Middleware is like a plugin that helps you modify the request or response in a specific way.

Example:

require 'faraday_middleware'

client =
  Faraday.new do |f|
    f.response :json
    f.adapter :net_http
  end

client.get('https://api.github.com/repos/vmg/redcarpet/issues')

This middleware will parse the response as JSON automatically.

If you want to send JSON instead of parsing it use f.request :json instead of f.response :json.

Faraday Video

Bonus: How to Debug Your HTTP Requests

If you want to see exactly what your code is sending so that you can make sure it’s working correctly…

…then I have a nice little trick for you.

You can use this one-liner:

ruby -rsocket -e "trap('SIGINT') { exit }; Socket.tcp_server_loop(8080) { |s,_| puts s.readpartial(1024); puts; s.puts 'HTTP/1.1 200'; s.close }

This creates a server on port 8080 that prints everything it receives, returns an HTTP status code & then closes the connection.

You can send requests against localhost:8080 to see them.

Like this:

Faraday.new('http://localhost:8080', headers: { 'User-Agent' => 'test', 'foo' => '1234' }).get

# GET / HTTP/1.1
# User-Agent: test
# Foo: 1234
# Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3
# Accept: */*
# Connection: close
# Host: localhost:8080

Summary

You learned how to send all kinds of HTTP request, starting from the basic GET to request information, to the POST request to submit information.

You have seen an overview of the most popular Ruby gems for working with HTTP requests. My favorite is faraday because of the flexibility it offers, but feel free to pick another gem if it fits your needs better.

If you enjoyed this post please share it with your friends so they can enjoy it too 🙂

Thanks for reading!

9 comments
Leonardo says a few months ago

An excellent article, very complete.

I use rest-client a lot 🙂
The Typhoeus I still did not know, very cool.
Thanks

    Jesus Castello says a few months ago

    Hey Leornado,
    I’m glad you found the article useful!

    Thanks for reading 🙂

Erik says a few months ago

Like it. Can you recomment a book containing ruby client and -server. Thanks your inspiration.
Erik.

obi says a few months ago

While I used to like Faraday too for the same reasons, I tend to like the “http” gem more these days, with its simple API. “httpx” is very similar, but handles pipelining and http2 better.

    Jesus Castello says a few months ago

    Thanks for sharing your experience 🙂

Ronaldo says a few months ago

Bookmarked! A good and useful approach.
Thank you Castello!

    Jesus Castello says a few months ago

    I’m glad you found it useful Ronaldo 🙂

Derrick says a couple of months ago

Nice post! HTTParty has always been my go-to but I might have to try some of the other ones you mentioned.

    Jesus Castello says a couple of months ago

    Thanks for reading! 🙂

Comments are closed