Posts

How to get href value of each a tag from the html using python Beautiful Soup

Image
import requests from bs4 import BeautifulSoup link = "http://www.flipkart.com/mobiles?otracker=hp_header_nmenu_sub_Electronics_0_Mobiles" doc = requests.get(link) soup = BeautifulSoup(doc.text, 'html.parser') main_div = soup.find(id="list-tagcloud") div2=main_div.find_all('div')[1] links = div2.find_all('a') for link in links:     print link.attrs.get('href') OUTPUT ====== /mobiles/motorola~brand/pr?sid=tyy,4io /mobiles/lenovo~brand/pr?sid=tyy,4io /mobiles/samsung~brand/pr?sid=tyy,4io /mobiles/leeco~brand/pr?sid=tyy,4io /yu-yunicorn/p/itmejeuf7egdedar?pid=MOBEJ3MF23Q9MGMH /mobiles/honor~brand/pr?sid=tyy,4io /mobiles/mi~brand/pr?sid=tyy,4io /mobiles/asus~brand/pr?sid=tyy,4io /mobiles/apple~brand/pr?sid=tyy,4io /mobiles/intex~brand/pr?sid=tyy,4io /mobiles/sony~brand/pr?sid=tyy,4io /mobiles/alcatel~brand/pr?sid=tyy,4io /mobiles/lava~brand/pr?sid=tyy,4io /gionee-store /mobiles/pr?sid=tyy,4io
Post a Comment
Read more

How to dispaly data from html tags using python Beautiful Soup

Image
import requests from bs4 import BeautifulSoup link = "http://www.flipkart.com/mobiles?otracker=hp_header_nmenu_sub_Electronics_0_Mobiles" doc = requests.get(link) soup = BeautifulSoup(doc.text, 'html.parser') main_div = soup.find(id="list-tagcloud") div2=main_div.find_all('div')[1] for x in div2.strings:     print x OUTPUT ====== Motorola Lenovo Samsung LeEco Yunicorn Honor Mi Asus Apple Intex Sony Alcatel Lava Gionee All
Post a Comment
Read more

Find all the URLs found within a page’s using python

Find all the URLs found within a page’s from bs4 import BeautifulSoup html_doc = """ <html><head><title>The Dormouse's story</title></head> <body> <p class="title"><b>The Dormouse's story</b></p> <p class="story">Once upon a time there were three little sisters; and their names were <a href="http://example.com/elsie" class="sister" id="link1">Elsie</a>, <a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and <a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>; and they lived at the bottom of a well.</p> <p class="story">...</p> """ soup = BeautifulSoup(html_doc, 'html.parser') for link in soup.find_all('a'):     print link.get('href...
Post a Comment
Read more

How to use BeautifulSoup in python

BeautifulSoup 4 in Python.Beautiful Soup is a Python library for pulling data out of HTML and XML files. html_doc = """ <html><head><title>The Dormouse's story</title></head> <body> <p class="title"><b>The Dormouse's story</b></p> <p class="story">Once upon a time there were three little sisters; and their names were <a href="http://example.com/elsie" class="sister" id="link1">Elsie</a>, <a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and <a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>; and they lived at the bottom of a well.</p> <p class="story">...</p> """ from bs4 import BeautifulSoup soup = BeautifulSoup(html_doc, 'html.parser') #prin...
Post a Comment
Read more

Status Codes

Note     1xx: Informational - Request received, continuing process     2xx: Success - The action was successfully received, understood, and accepted     3xx: Redirection - Further action must be taken in order to complete the request     4xx: Client Error - The request contains bad syntax or cannot be fulfilled     5xx: Server Error - The server failed to fulfill an apparently valid request         Available Formats Value     Description    100     Continue 101     Switching Protocols 102     Processing 103-199     Unassigned     200     OK 201     Created 202     Accepted 203     Non-Authoritative Information 204     No Content 205     Reset Content 206     Parti...
Post a Comment
Read more

How to write simple unit test in django

Success Run test file ============== tests.py ===== from django.test import TestCase # Create your tests here. class TestStringMethods(TestCase):     def test_upper(self):         self.assertEqual('foo'.upper(), 'FOO')     def test_isupper(self):         self.assertTrue('FOO'.isupper())         self.assertFalse('Foo'.isupper()) Success Run test file ============== $./manage.py test Creating test database for alias 'default'... .. ---------------------------------------------------------------------- Ran 2 tests in 0.664s OK Destroying test database for alias 'default'... --------------------------- Failed Run test file ============ tests.py ===== from django.test import TestCase # Create your tests here. class TestStringMethods(TestCase):     def test_upper(self):      ...
Post a Comment
Read more

How to write logging in django

settings.py ------------------ # https://docs.python.org/2/library/logging.html#logrecord-attributes   LOGGING = {     'version': 1,     'disable_existing_loggers': False,     'formatters':{         'details':{             'format':'%(asctime)s %(process)d %(filename)s %(funcName)s %(lineno)d %(levelname)s %(message)s'         },     },     'handlers': {         'file': {             'level': 'DEBUG',             'class': 'logging.FileHandler',             'filename': 'location for/ log file/debug.log',             'formatter': '...
Post a Comment
Read more