How to get href value of each a tag from the html using python Beautiful Soup

import requests
from bs4 import BeautifulSoup

link = "http://www.flipkart.com/mobiles?otracker=hp_header_nmenu_sub_Electronics_0_Mobiles"
doc = requests.get(link)
soup = BeautifulSoup(doc.text, 'html.parser')
main_div = soup.find(id="list-tagcloud")
div2=main_div.find_all('div')[1]
links = div2.find_all('a')
for link in links:
    print link.attrs.get('href')


OUTPUT
======
/mobiles/motorola~brand/pr?sid=tyy,4io
/mobiles/lenovo~brand/pr?sid=tyy,4io
/mobiles/samsung~brand/pr?sid=tyy,4io
/mobiles/leeco~brand/pr?sid=tyy,4io
/yu-yunicorn/p/itmejeuf7egdedar?pid=MOBEJ3MF23Q9MGMH
/mobiles/honor~brand/pr?sid=tyy,4io
/mobiles/mi~brand/pr?sid=tyy,4io
/mobiles/asus~brand/pr?sid=tyy,4io
/mobiles/apple~brand/pr?sid=tyy,4io
/mobiles/intex~brand/pr?sid=tyy,4io
/mobiles/sony~brand/pr?sid=tyy,4io
/mobiles/alcatel~brand/pr?sid=tyy,4io
/mobiles/lava~brand/pr?sid=tyy,4io
/gionee-store
/mobiles/pr?sid=tyy,4io


Comments

Post a Comment

Popular posts from this blog

AttributeError: Got AttributeError when attempting to get a value for field `abc` on serializer `PfleSerializer`. The serializer field might be named incorrectly and not match any attribute or key on the `QuerySet` instance. Original exception text was: 'QuerySet' object has no attribute 'abc'.

AttributeError: Got AttributeError when attempting to get a value for field `abc` on serializer ` PfleSerializer `. The serializer field might be named incorrectly and not match any attribute or key on the `QuerySet` instance. Original exception text was: 'QuerySet' object has no attribute 'abc'. >>> p=Pfle.objects.all() >>> seri=PfleSerializer(p,many=True) >>> seri.data
Read more

ImportError: No module named regex

$ ./topy.py /home/sanu/horizon/horizon/view.py Traceback (most recent call last): File "./topy.py", line 21, in <module> import regex ImportError: No module named regex $ sudo apt-get install python-dev [sudo] password for sanu: Reading package lists... Done Building dependency tree Reading state information... Done The following extra packages will be installed: libexpat1-dev libpython-dev libpython2.7-dev python2.7-dev The following NEW packages will be installed: libexpat1-dev libpython-dev libpython2.7-dev python-dev python2.7-dev 0 upgraded, 5 newly installed, 0 to remove and 0 not upgraded. Need to get 22.4 MB of archives. After this operation, 35.1 MB of additional disk space will be used. Do you want to continue? [Y/n] y Get:1 http://in.archive.ubuntu.com/ubuntu/ trusty/main libexpat1-dev amd64 2.1.0-4ubuntu1 [115 kB] Get:2
Read more