For Web scraping, we need to install a WebDriver and some python related software.
WebDriver is an open source tool for automated testing of webapps across many browsers. It provides capabilities for navigating to web pages, user input, JavaScript execution, and more. ChromeDriver is a standalone server which implements WebDriver’s wire protocol for Chromium.
pi@raspberrypi:~/Zpython3 $ sudo apt-get install chromium-browser pi@raspberrypi:~/Zpython3 $ sudo apt-get install chromium-chromedriver pi@raspberrypi:~/Zpython3 $ /usr/lib/chromium-browser/chromium-browser --version Chromium 65.0.3325.181 pi@raspberrypi:~/Zpython3 $ /usr/lib/chromium-browser/chromedriver --version ChromeDriver 2.35 (0) pi@raspberrypi:~/Zpython3 $ wget https://github.com/jjhelmus/berryconda/releases/download/v2.0.0/Berryconda3-2.0.0-Linux-armv7l.sh pi@raspberrypi:~/Zpython3 $ which python3 /home/pi/berryconda3/bin/python3 pi@raspberrypi:~/Zpython3 $ python3 -V Python 3.6.1 pi@raspberrypi:~/Zpython3 $ pip3 list Package Version ------------ --------- selenium 3.141.0 pi@raspberrypi:~/Zpython3 $ pip3 show selenium Name: selenium Version: 3.141.0 Summary: Python bindings for Selenium Home-page: https://github.com/SeleniumHQ/selenium/ Author: UNKNOWN Author-email: UNKNOWN License: Apache 2.0 Location: /home/pi/berryconda3/lib/python3.6/site-packages Requires: urllib3 Required-by: pi@raspberrypi:~/Zpython3 $ python3 selenium01.py
import time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
chrome_option = webdriver.ChromeOptions()
chrome_option.add_argument('--headless')
driver = webdriver.Chrome('/usr/lib/chromium-browser/chromedriver', chrome_options=chrome_option)
for irep in range(360):
driver.get('https://pskreporter.info/pskmap.html?preset&callsign=ja1a&band=6000000-8000000&timerange=86400')
assert 'Display Reception Reports' in driver.title
time.sleep(200)
fname = time.strftime("%a%d%b%Y%H%M%S.png", time.gmtime())
print(fname)
driver.save_screenshot(fname)
time.sleep(100)
driver.quit()