v4l3n71n/web_scrap
1
0
Fork 0
Code Issues Pull Requests Projects Releases 10 Wiki Activity

web_scap #1

Merged
v4l3n71n merged 11 commits from web_scap into master 2023-03-07 11:28:15 +00:00
Conversation 0 Commits 11 Files Changed 1 +6 -1
1 changed files with 6 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files
Showing only changes of commit d21af4f60a - Show all commits

7
web_scrap.py
View File

@ -18,6 +18,7 @@ if page.status_code == 200:
if href != "#":
page_url.append(href)
webpage = []
for i in page_url:
page = requests.get(i)
if page.status_code == 200:
@ -43,7 +44,11 @@ for i in page_url:
soup = BeautifulSoup(page.text, 'html.parser')
h2 = soup.find_all("h2")
for title in h2:
print(title.find_all("a")[0].get("href", "/"))
href = title.find_all("a")[0].get("href", "/")
if href not in webpage:
webpage.append(href)
print(webpage)