100 thoughts on “Python Requests Tutorial: Request Web Pages, Download Images, POST Data, Read JSON, and More

  1. My next video will be a real-world example of a script I wrote to monitor my personal website. We'll use the Request library to monitor the site, and if it is down then we will learn how to send an email and automatically restart the server. I hope everyone finds this useful! Hope you're having a great week!

  2. Can you please make a video about threads? I'd like to combine requests and threads to access a web page and get data from it, like downloading multiple pictures at the same time.

  3. Hi Corey,

    You started Kali video series but you only did one video. Are you going to resume this anytime soon, also, which is the best way to get in contact with you

  4. corey can you give a real-world example of when we will use passing parameters with get request and also of basic auth using auth tuple

  5. Great job on explaining this! Your tutorials are to the point, clear and cover everything we need to know. Every video takes me to the next level.

  6. Anyone who watches your videos without hitting the like button would certainly feel guilty and is indeed guilty.
    Thanks

  7. Hey Corey, thanks for this guide.
    In work sometimes `allow_redirects=False` as request param is helpful as well.
    Also if your request resulted in several redirects (which is also quite usual), request.history will return all request objects which have happened.

  8. Hi Corey, Your videos have become my one source of all the information i need on python, your videos are simple and just great!, thank you very much.

  9. Do you have videos about Python Requests? that fill a form data table? also in CRUD. I'm beginner in python.

  10. Thank you very much for the good "no the great" content. I make sure to like every video i have my eyes on.
    I have had the honor to make my first pull request to one of your tutorials l, and you actually accepted it :).
    I have a question that i would like to hear your input about.
    I made a django website and it now lives on a hosted server, this site send people Texts (SMSs) when certain tasks are finished by our employees. (Btw Using twilio)
    I am writing the info of the SMSs to t a table in my database. I have a script (sms_worker) that when triggered goes through them and send the 'new' SMSs.
    I want this to br running 24/7.
    I way i got around it, is that i wrote a 'requestes' script that will run every 20 seconds to trigger the 'sms_worker' using time.wait().
    It works just fine but i feel this is a dirty workaround. I would like your input about how to run it in a more consistent way.

    So sorry about the long comment

  11. WOW, the 100th video in your MARVELOUS Python tutorial series with my 100th thumbs up. Thank u Corey, you are truly amazing!

  12. Hey Corey, this was an awesome video intro to the requests package!

    I have a question that's been killing me. Currently I'm working on a webscraper for collecting pdfs of equities research reports on a set of stocks. I have access to the site where they're located, but the url doesn't end in .pdf. I've heard people say on Stack Overflow that pdf urls need to end in .pdf for requests to recognize the data. Is there any way to manually pass in the file extension to requests, so that it knows that we're trying to get a pdf? I've tried downloading them also with wget, and in both instances, they download somehow damaged where Acrobat Reader will not open them.

    Thanks for all your hard work, the videos have been amazing!

  13. Hey Corey, just wanted to tell you what I realized recently. 
    Whenever I'm reading a tutorial or some explanations in python/web dev area, I read it with your voice :D.
    Thanks for the great work and please don't stop 🙂

  14. Your video is a time saver. The information which we need to process from more than one blog or resource is presented elegantly and beautifully to us. Thank you Corey.

  15. I am applying this tutorial to my real world project. However, I feel I need more explanation about data modeling like proxy model, multi-inheritance, multi-to-multi relationship …

  16. U haven't covered the redirect part and i saw the notification part to covert the video to hindi language, why?

  17. hey corey m a big fan of yours tut much love bro
    n please make tut on the google Oauth plzzz i want to add it to ma project and i am fedup of seaching this content using Python plzzz thanks in advance <3 <3

  18. Hi Corey, I'm a business intelligence developer and a I'm just starting with python. I get asked a lot to get data out of API's. Is the request lib the best place to start or are there better lib's out there. Like I said, I'm very green when it comes to python so any advice would help. Thanks

  19. Fab tutorial ❤️. But have a question.
    How will you emulate the behavior of a "click" or press button with requsts

  20. Corey you are the best trainer or develper for me as when i see your videos ill get both new things and the things in my mind that i wish to ask you but you answer them in your video itself. so you are trainer and develper for me.keep the good work cheers.

  21. Hi Corey
    Can you please tell me how can we verify post data from server side ? How can we check it from server/url not from python script i.e. req.text ?

  22. @corey This is an awesome video on requests.. If anybody asks me what is the best place to learn python, i will directly point them to your channel.

    Can you also cover some indepth topics of this "requests" module?

  23. First of all, Thanks a ton Corey for your superb videos, I am a beginner for Python and I loved watching your videos. I credit you for the progress that I have made in Python. I have a query, I am planning to monitor a website at work but it has OKTA authentication and I am unable to get through it. Is there any way you could help with this scenario? Appreciate your inputs! 🙂

  24. Harvard should hire you. I learned so much from watching your videos. Much more through than than CS50 course in web development(although it is a great series). Thank you, your hard work is very much appreciated. I will be donating and sharing.

  25. The 6 dislikes are from some drunk people who missclicked I think.

    WHO WOULD DISLIKE A YOUTUBER LIKE YOU? NOONE SO SOME RANDOM DUDES MISSCLICKED

  26. When I try to download image its giving me an error:
    SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated UXXXXXXXX escape
    What should I do?

  27. Hey Corey, how can I pass by these members only auth to scrap the contacts? I have the user and passwd, I already used Requests, BeautifullSoup and Pandas to scrape and create an organized dataframe, but it keeps coming members only instead of the contacts. Can you help me with that? The site is https://www.wcaworld.com/Directory/Members/63846#login

  28. Great tutorial Corey. Really like your stuff – thank you for covering the material at a brisk pace. Loads of excellent, well presented content without having to listen to slow tedious simplifications. Awesome!

  29. Lets all remember that in the world of the current web with REACT and JS…Scraping without a driver is useless QQ.

  30. so what I don't get is how all of these modules magically do all of this stuff. I know that modules are also can kinda be called big variables and you use these tools (modules) to builds your own house (code). I checked the git hub for the library and nothing is making sense, how does this module actually work or any module per say.

  31. I am really looking forward to seeing a video about threading. I hope you will make it as well! Thanks for great tutorials even better than paid courses around in my opinion! I'd rather support your channel instead of paying for less quality courses.

  32. Tx!! You just solved some of my problems !!!:-)
    As a noob i try to understand how to post a json to a webhook. I think i start to understand how post/get works and the syntax surounding it looks like.
    Well explained!:-)
    // The Noob

  33. Ok, Corey… Corey-Corey-Corey…..*ominous silence"
    You are the Man. Hands down the most well-put and thought through videos on Python I've seen on Youtube.
    Much appreciated! I thank thee for thy work for tis magnificent

  34. Hello Corey,, have been following you for quiet a while now,, really helpful in overall understanding of the subject in hands,, however, this is the first clip where I don't understand what's happening exactly, and obviously it's my problem, so wanted to ask about what prerequisites are needed in order to follow along in the video,

    again man many thanks for the effort put here,, you are one of a kind

  35. 09:57 " I never understand how some of these people are so productive and…"
    Since, I started watching your channel I have been thinking the exact same thing about you Mr. Schafer. Honestly, I could never understand how you could author so many impeccable videos on programming so fluently. You are giving so much to people like me. Thank you from the bottom of my heart 🙂

  36. Why didn't somebody tell me about the dir float(I hope that is the right word. I am still learning) when I as first starting python? That is the most useful thing!

  37. Great content! I have a problem though: I'm trying to login as you suggested but in my case, when I do the inspection of the website in Network/Headers I don't find anything about "Form Data" I don't find anything about password or username. Is it another kind of login or am I doing something wrong? Thank you in advance.

  38. Would you cover pagination when doing an API request and how to handle the multiple pages of JSON that is returned? Thanks for your videos.

  39. how to access this site, as it brings a banner in first visit through browser,
    https://www.dibbs.bsm.dla.mil/Rfq/RfqDates.aspx?category=recent

  40. at 3:08 I use your code and get this error:
    ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)

Leave a Reply

Your email address will not be published. Required fields are marked *