scrapy, cómo separar texto dentro de un elemento de etiqueta HTML Desarrollo de Python

**Quiero tener

nombre del lugar: Century Square, Liat Towers

Dirección: 2 Tampines Central 5, 541 torres Liat # 01-01

Código postal: Singapur 529509, Singapur 238888

Horario de apertura: 7-12 am, 24 horas al día **

Por ejemplo:

la primera en ” tiene 3 datos que quiero (nombre, dirección, postal). ¿Cómo los divido?

Aquí está mi código de araña:

 from scrapy.spider import BaseSpider from scrapy.selector import HtmlXPathSelector import re from todo.items import wendyItem class wendySpider(BaseSpider): name = "wendyspider" allowed_domains = ["wendys.com.sg"] start_urls = ["http://www.wendys.com.sg/outlets.php"] def parse(self, response): hxs = HtmlXPathSelector(response) values = hxs.select('//td') items = [] for value in values: item = wendyItem() item['name'] = value.select('//span[@class="foodTitle"]/text()').extract() item['address'] = value.select().extract() item['postal'] = value.select().extract() item['hours'] = value.select().extract() item['contact'] = value.select().extract() items.append(item) return items

Seleccionaría todos los

que contienen un

 //div[@id="menu_list"]//td[@valign="top"][.//span[@class="foodTitle"]]

y luego para cada una de estas celdas td , obtenga todos los nodos de texto

 .//text()

Obtienes algo así:

 ['\n ', '\n ', 'Century Square', '\n 2 Tampines Central 5', '\n #01-44-47 Century Square', '\n Singapore 529509', '\n ', '\n ', 'Opening Hours:', u'\n 7am to 12am (Sun-Thu &\xa0PH)', u'\n 24 Hours (Fri & Sat\xa0&', '\n ', '\n Eve of PH)', '\n Telephone: 6789 0457', '\n ']

 ['\n ', 'Liat Towers', '\n 541 Liat towers #01-01', '\n Orchard Road', '\n Singapore 238888', 'Opening Hours: ', '\n 24 hours (Daily)', '\n Telephone: 6737 8036']

Algunos de estos nodos de texto tienen una representación de cadena que es todo espacio en blanco, así que elimínelos y busque las palabras clave “Horario de apertura” y “Teléfono” para procesar las líneas en un bucle:

 from scrapy.spider import BaseSpider from scrapy.selector import HtmlXPathSelector import re from todo.items import wendyItem class wendySpider(BaseSpider): name = "wendyspider" allowed_domains = ["wendys.com.sg"] start_urls = ["http://www.wendys.com.sg/outlets.php"] def parse(self, response): hxs = HtmlXPathSelector(response) cells = hxs.select('//div[@id="menu_list"]//td[@valign="top"][.//span[@class="foodTitle"]]') items = [] for cell in cells: item = wendyItem() # get all text nodes # some lines are blank so .strip() them lines = cell.select('.//text()').extract() lines = [l.strip() for l in lines if l.strip()] # first non-blank line is the place name item['name'] = lines.pop(0) # for the other lines, check for "Opening hours" and "Telephone" # to store lines in correct list container address_lines = [] hours_lines = [] telephone_lines = [] opening_hours = False telephone = False for line in lines: if 'Opening Hours' in line: opening_hours = True elif 'Telephone' in line: telephone = True if telephone: telephone_lines.append(line) elif opening_hours: hours_lines.append(line) else: address_lines.append(line) # last address line is the postal code + town name item['address'] = "\n".join(address_lines[:-1]) item['postal'] = address_lines[-1] # ommit "Opening hours" (first element in list) item['hours'] = "\n".join(hours_lines[1:]) item['contact'] = "\n".join(telephone_lines) items.append(item) return items

Century Square
2 Tampines Central 5
#01-44-47 Century Square
Singapore 529509

Opening Hours:
7am to 12am (Sun-Thu & PH)
24 Hours (Fri & Sat &

Eve of PH)
Telephone: 6789 0457

Liat Towers
541 Liat towers #01-01
Orchard Road
Singapore 238888

Opening Hours:
24 hours (Daily)

Telephone: 6737 8036