Python requests Module

Python requests is a commonly used HTTP request library that makes it easy to send HTTP requests to websites and retrieve response results.

The requests module is simpler than the urllib module.

To use requests to send HTTP requests, you need to first import the requests module:

import requests

After importing, you can send HTTP requests using the methods provided by requests to a specified URL, for example:

Example

# Import the requests package
import requests

# Send a request
x = requests.get('https://www.runoob.com/')

# Return the web page content
print(x.text)

Each time you call a requests method, it returns a response object containing specific response information such as status code, response headers, response content, etc.:

print(response.status_code)  # Get the response status code
print(response.headers)  # Get the response headers
print(response.content)  # Get the response content

More response information is as follows:


Property or Method	Description
apparent_encoding	Encoding method
close()	Close the connection to the server
content	Returns the response content in bytes
cookies	Returns a CookieJar object containing the cookies sent back from the server
elapsed	Returns a timedelta object containing the time elapsed between sending the request and receiving the response, useful for testing response speed. For example, r.elapsed.microseconds indicates how many microseconds the response took.
encoding	The encoding method for decoding r.text
headers	Returns the response headers in dictionary format
history	Returns a list of response objects containing the request history (urls)
is_permanent_redirect	Returns True if the response is a permanently redirected URL, otherwise False
is_redirect	Returns True if the response was redirected, otherwise False
iter_content()	Iterates over the response
iter_lines()	Iterates over the response lines
json()	Returns the result as a JSON object (the result must be in JSON format, otherwise an error will be raised)
links	Returns the parsed header links of the response
next	Returns the PreparedRequest object for the next request in the redirect chain
ok	Checks the value of “status_code”. Returns True if it is less than 400, otherwise False
raise_for_status()	If an error occurs, the method returns an HTTPError object
reason	The description of the response status, such as “Not Found” or “OK”
request	Returns the request object that requested this response
status_code	Returns the HTTP status code, such as 404 and 200 (200 is OK, 404 is Not Found)
text	Returns the response content as unicode type data
url	Returns the URL of the response

Example

# Import the requests package
import requests

# Send a request
x = requests.get('https://www.runoob.com/')

# Return the HTTP status code
print(x.status_code)

# Description of the response status
print(x.reason)

# Return the encoding
print(x.apparent_encoding)

The output is as follows:

200
OK
utf-8

Request a JSON data file and return the JSON content:

Example

# Import the requests package
import requests

# Send a request
x = requests.get('https://www.runoob.com/try/ajax/json_demo.json')

# Return JSON data
print(x.json())

The output is as follows:

{'name': 'Website', 'num': 3, 'sites': [{'name': 'Google', 'info': ['Android', 'Google Search', 'Google Translate']}, {'name': 'Runoob', 'info': ['Runoob Tutorial', 'Runoob Tools', 'Runoob WeChat']}, {'name': 'Taobao', 'info': ['Taobao', 'Online Shopping']}]}

requests Methods

The requests methods are as shown in the table below:


Method	Description
delete(url, args)	Sends a DELETE request to the specified URL
get(url, params, args)	Sends a GET request to the specified URL
head(url, args)	Sends a HEAD request to the specified URL
patch(url, data, args)	Sends a PATCH request to the specified URL
post(url, data, json, args)	Sends a POST request to the specified URL
put(url, data, args)	Sends a PUT request to the specified URL
request(method, url, args)	Sends the specified request method to the specified URL

Using requests.request() to send a GET request:

Example

# Import the requests package
import requests

# Send a request
x = requests.request('get', 'https://www.runoob.com/')

# Return the web page content
print(x.status_code)

The output is as follows:

Setting request headers:

Example

# Import the requests package
import requests

 
kw = {'s':'python tutorial'}

# Set request headers
headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36"}
 
# params accepts a dictionary or string of query parameters, dictionary type is automatically converted to URL encoding, no need for urlencode()
response = requests.get("https://www.runoob.com/", params = kw, headers = headers)

# Check the response status code
print (response.status_code)

# Check the response header character encoding
print (response.encoding)

# Check the full URL address
print (response.url)

# Check the response content, response.text returns data in Unicode format
print(response.text)

The output is as follows:

200
UTF-8
https://www.runoob.com/?s=python+tutorial

... other content...

The post() method sends a POST request to the specified URL. The general format is as follows:

requests.post(url, data={key: value}, json={key: value}, args)

url is the request URL.
data parameter is the dictionary, tuple list, bytes, or file object to send to the specified URL.
json parameter is the JSON object to send to the specified URL.
args are other parameters such as cookies, headers, verify, etc.

Example

# Import the requests package
import requests

# Send a request
x = requests.post('https://www.runoob.com/try/ajax/demo_post.php')

# Return the web page content
print(x.text)

The output is as follows:

<p style='color:red;'>This content was requested using the POST method.</p><p style='color:red;'>Request time:
2022-05-26 17:30:47</p>

POST request with parameters:

Example

# Import the requests package
import requests

# Form parameters with parameter names fname and lname
myobj = {'fname': 'RUNOOB','lname': 'Boy'}

# Send a request
x = requests.post('https://www.runoob.com/try/ajax/demo_post2.php', data = myobj)

# Return the web page content
print(x.text)

The output is as follows:

<p style='color:red;'>Hello, RUNOOB Boy, how are you doing today?</p>

Additional Request Parameters

When sending requests, we can attach additional parameters such as request headers, query parameters, request body, etc., for example:

headers = {'User-Agent': 'Mozilla/5.0'}  # Set request headers
params = {'key1': 'value1', 'key2': 'value2'}  # Set query parameters
data = {'username': 'example', 'password': '123456'}  # Set request body
response = requests.post('https://www.runoob.com', headers=headers, params=params, data=data)

The above code sends a POST request and attaches request headers, query parameters, and a request body.

In addition to basic GET and POST requests, requests also supports other HTTP methods such as PUT, DELETE, HEAD, OPTIONS, etc.