Python Web Scraping: Sending User-Agent Headers with Requests

You can send a User-Agent header in your HTTP request by using the requests library in Python. Here is an example:

import requests

url = 'https://example.com'
headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3'}

response = requests.get(url, headers=headers)

print(response.content)

In the above example, we create a dictionary headers that contains a key-value pair for the User-Agent header. We then pass this dictionary to the get method of the requests library, along with the URL we want to fetch. This will send a GET request to the specified URL with the User-Agent header set to the value we provided. The response content is then printed to the console.

Note that the User-Agent header can be set to any value you want, but it is generally a good idea to use a well-known user agent string so that the server knows what kind of client is sending the request.