arrow_upward
Webscraping Javascript through Python
Anonymous57928
Junior Member
Anonymous57928
Junior Member
**
forum
Posts:1
chat
Threads:1
calendar_today
Joined:May 2019
thumb_up
Reputation: 1
expand_more
#1
15-05-2019, 12:23 AM
Guys! I'm seeking to create a webscraper that is able to scrape text from Javascript sources. Any tips or tricks? Even though I am able to scrape the first page via BeautifulSoup, I am not able to go to the second page.

Just let me know.

Cheers!



[+] 1 user Likes Anonymous57928's post
stevenyyy
Junior Member
stevenyyy
Junior Member
**
forum
Posts:20
chat
Threads:0
calendar_today
Joined:Nov 2019
thumb_up
Reputation: 0
expand_more
#2
30-11-2019, 08:31 AM
selenium can do lots of scraping, but it not easy



Foobarbaz
Onyx Team
Foobarbaz
Onyx Team
Onyx
forum
Posts:12
chat
Threads:0
calendar_today
Joined:Jun 2019
thumb_up
Reputation: 5
expand_more
#3
23-01-2020, 07:32 PM
Selenium and chrome driver .. you need to run an headless browser to scrape running JavaScript



spaghets
Junior Member
spaghets
Junior Member
**
forum
Posts:7
chat
Threads:2
calendar_today
Joined:Apr 2021
thumb_up
Reputation: 5
expand_more
#4
13-04-2021, 12:03 AM
Selenium will work with a driver to interact where dynamic interaction/loading is required.
It depends on how static the site is. BeautifulSoup can do alright but you should also take a look at ScraPy, its had some automation added recently from when it first started

RE: Webscraping Javascript through Python.

Registered Members Only

You need to be a registered member to see more on RE: Webscraping Javascript through Python.
Login or Sign up to get access to a huge variety of top quality leaks.