beautifulsoup
bs4 模块
BeautifulSoup 也是一个HTML/XML 解释器。
相对于lxml只会局部遍历，BeautifulSoup是基于HTML DOM(Document Object Model)， 会加载整个文档，解析整个DOM树，因此时间和内存开销都会大很多
支持CSS选择器，XML解析器
安装
解析器
推荐使用lxml，效率更高
html5lib 可以自动补全标签
| 解析器 |
使用方法 |
优势 |
劣势 |
| Python标准库 |
BeautifulSoup(markup, "html.parser") |
Python的内置标准库执行速度适中文档容错能力强 |
Python 2.7.3 or 3.2.2)前 的版本中文档容错能力差 |
| lxml HTML 解析器 |
BeautifulSoup(markup, "lxml") |
速度快文档容错能力强 |
需要安装C语言库 |
| lxml XML 解析器 |
BeautifulSoup(markup, ["lxml", "xml"])``BeautifulSoup(markup, "xml") |
速度快唯一支持XML的解析器 |
需要安装C语言库 |
| html5lib |
BeautifulSoup(markup, "html5lib") |
最好的容错性以浏览器的方式解析文档生成HTML5格式的文档 |
速度慢不依赖外部扩展 |
对象的种类
Beautiful Soup将复杂HTML文档转换成一个复杂的树形结构,每个节点都是Python对象,所有对象可以归纳为4种: Tag , NavigableString , BeautifulSoup , Comment .
Tag
BeautifulSoup 中所有的标签都是tag类型，并且BeautifulSoup的对象其实本质上也是一个Tag类型。
所以find，find_all并不是BeatifulSoup的，而是Tag的
soup = BeautifulSoup('<b class="boldest">Extremely bold</b>')
tag = soup.b
type(tag)
# <class 'bs4.element.Tag'>
Name
每个tag都有自己的名字,通过 .name 来获取:
如果改变了tag的name,那将影响所有通过当前Beautiful Soup对象生成的HTML文档:
tag.name = "blockquote"
tag
# <blockquote class="boldest">Extremely bold</blockquote>
Attributes
一个tag可能有很多个属性. tag <b class="boldest"> 有一个 “class” 的属性,值为 “boldest” . tag的属性的操作方法与字典相同:
tag['class']
# u'boldest'
也可以直接”点”取属性, 比如: .attrs :
tag.attrs
# {u'class': u'boldest'}
tag的属性可以被添加,删除或修改. 再说一次, tag的属性操作方法与字典一样
tag['class'] = 'verybold'
tag['id'] = 1
tag
# <blockquote class="verybold" id="1">Extremely bold</blockquote>
del tag['class']
del tag['id']
tag
# <blockquote>Extremely bold</blockquote>
tag['class']
# KeyError: 'class'
print(tag.get('class'))
# None
多值属性
最常见的多值的属性是 class (一个tag可以有多个CSS的class)
在Beautiful Soup中多值属性的返回类型是list:
css_soup = BeautifulSoup('<p class="body strikeout"></p>')
css_soup.p['class']
# ["body", "strikeout"]
css_soup = BeautifulSoup('<p class="body"></p>')
css_soup.p['class']
# ["body"]
NavigableString （字符串遍历）
字符串常被包含在tag内.Beautiful Soup用 NavigableString 类来包装tag中的字符串
继承自Python中的str，使用起来和stg是一样的
string
获取当前某个标签下的非标签字符串
strings
获取某个标签下的子孙非标签字符串
trs = soup.find_all('tr')
for tr in trs:
infos = list(tr.strings)
stripped_strings
获取某个标签下的子孙非标签字符串，会去掉空白字符
trs = soup.find_all('tr')
for tr in trs:
infos = list(tr.stripped_strings)
BeautifulSoup
BeautifulSoup 对象并不是真正的HTML或XML的tag,所以它没有name和attribute属性.但有时查看它的 .name 属性是很方便的,所以 BeautifulSoup 对象包含了一个值为 “[document]” 的特殊属性 .name
继承自Tag，用来生成BeautifulSoup 树，对于find_all，select查找方法，其实还是Tag的
soup.name
# u'[document]'
Tag , NavigableString , BeautifulSoup 几乎覆盖了html和xml中的所有内容,但是还有一些特殊对象.容易让人担心的内容是文档的注释部分:
继承自NavigableString
markup = "<b><!--Hey, buddy. Want to buy a used parser?--></b>"
soup = BeautifulSoup(markup)
comment = soup.b.string
type(comment)
# <class 'bs4.element.Comment'>
遍历HTML
html 文件
html_doc = """
<html><head><title>The Dormouse's story</title></head>
<p class="title"><b>The Dormouse's story</b></p>
<p class="story">Once upon a time there were three little sisters; and their names were
<a href="http://example.com/elsie" class="sister" id="link1">Elsie</a>,
<a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and
<a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>;
and they lived at the bottom of a well.</p>
<p class="story">...</p>
"""
通过tag的名字
操作文档树最简单的方法就是告诉它你想获取的tag的name.如果想获取
标签,只要用
soup.head :
soup.head
# <head><title>The Dormouse's story</title></head>
soup.title
# <title>The Dormouse's story</title>
.parent
通过 .parent 属性来获取某个元素的父节点.在例子“爱丽丝”的文档中,
标签是
标签的父节点:</p>
<div class="hlcode"><pre><span class="n">title_tag</span> <span class="o">=</span> <span class="nx">soup.title</span>
<span class="nx">title_tag</span>
<span class="err">#</span> <span class="o"><</span><span class="nb">title</span><span class="o">></span><span class="nx">The</span> <span class="nx">Dormouse</span><span class="s1">'s story</title></span>
<span class="s1">title_tag.parent</span>
<span class="s1"># <head><title>The Dormouse'</span><span class="nb">s</span> <span class="nx">story</span><span class="o"></</span><span class="nb">title</span><span class="o">></</span><span class="nb">head</span><span class="o">></span>
</pre></div>
<p>文档的顶层节点比如<html>的父节点是 <code>BeautifulSoup</code> 对象:</p>
<div class="hlcode"><pre><span class="n">html_tag</span> <span class="o">=</span> <span class="n">soup</span><span class="p">.</span><span class="n">html</span>
<span class="n">type</span><span class="p">(</span><span class="n">html_tag</span><span class="p">.</span><span class="n">parent</span><span class="p">)</span>
<span class="cp"># <class 'bs4.BeautifulSoup'></span>
</pre></div>
<p><code>BeautifulSoup</code> 对象的 <code>.parent</code> 是None:</p>
<div class="hlcode"><pre><span class="n">print</span><span class="p">(</span><span class="n">soup</span><span class="p">.</span><span class="n">parent</span><span class="p">)</span>
<span class="cp"># None</span>
</pre></div>
<h2 id="parents">.parents</h2>
<p>通过元素的 <code>.parents</code> 属性可以递归得到元素的所有父辈节点,下面的例子使用了 <code>.parents</code> 方法遍历了<a>标签到根节点的所有节点.</p>
<div class="hlcode"><pre><span class="k">link</span> <span class="o">=</span> <span class="nx">soup.a</span>
<span class="k">link</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/elsie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link1"</span><span class="o">></span><span class="nx">Elsie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span>
<span class="nb">for</span> <span class="n">parent</span> <span class="k">in</span> <span class="k">link</span><span class="bp">.</span><span class="nx">parents</span><span class="p">:</span>
<span class="k">if</span> <span class="k">parent</span> <span class="nx">is</span> <span class="kc">None</span><span class="p">:</span>
<span class="nx">print</span><span class="p">(</span><span class="k">parent</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="nx">print</span><span class="p">(</span><span class="k">parent</span><span class="bp">.</span><span class="nb">name</span><span class="p">)</span>
<span class="err">#</span> <span class="nx">p</span>
<span class="err">#</span> <span class="nb">body</span>
<span class="err">#</span> <span class="nx">html</span>
<span class="err">#</span> <span class="err">[</span><span class="nb">document</span><span class="cp">]</span>
# None
</pre></div>
<h2 id="contents">.contents</h2>
<p>tag的 <code>.contents</code> 属性可以将tag的子节点以列表的方式输出:</p>
<div class="hlcode"><pre><span class="n">head_tag</span> <span class="o">=</span> <span class="nx">soup.head</span>
<span class="nx">head_tag</span>
<span class="err">#</span> <span class="o"><</span><span class="nb">head</span><span class="o">><</span><span class="nb">title</span><span class="o">></span><span class="nx">The</span> <span class="nx">Dormouse</span><span class="s1">'s story</title></head></span>
<span class="s1">head_tag.contents</span>
<span class="s1">[<title>The Dormouse'</span><span class="nb">s</span> <span class="nx">story</span><span class="o"></</span><span class="nb">title</span><span class="o">></span><span class="cp">]</span>
title_tag = head_tag.contents<span class="cp">[</span><span class="mi">0</span><span class="cp">]</span>
title_tag
# <span class="nt"><title></span>The Dormouse's story<span class="nt"></title></span>
title_tag.contents
# <span class="cp">[</span><span class="nx">u</span><span class="s1">'The Dormouse'</span><span class="nb">s</span> <span class="nx">story</span><span class="s1">']</span>
</pre></div>
<h2 id="children">.children</h2>
<p>通过tag的 <code>.children</code> 生成器,可以对tag的子节点进行循环:</p>
<div class="hlcode"><pre><span class="k">for</span> <span class="n">child</span> <span class="n">in</span> <span class="n">title_tag</span><span class="p">.</span><span class="n">children</span><span class="o">:</span>
<span class="n">print</span><span class="p">(</span><span class="n">child</span><span class="p">)</span>
<span class="err">#</span> <span class="n">The</span> <span class="n">Dormouse</span><span class="err">'</span><span class="n">s</span> <span class="n">story</span>
</pre></div>
<h2 id="next_sibling">.next_sibling</h2>
<p>在文档树中,使用 <code>.next_sibling</code> 和 <code>.previous_sibling</code> 属性来查询兄弟节点:</p>
<div class="hlcode"><pre><span class="nx">sibling_soup.b.next_sibling</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">c</span><span class="o">></span><span class="nx">text2</span><span class="o"></</span><span class="nx">c</span><span class="o">></span>
</pre></div>
<p>实际文档中的tag的 <code>.next_sibling</code> 和 <code>.previous_sibling</code> 属性通常是字符串或空白</p>
<div class="hlcode"><pre><span class="k">link</span> <span class="o">=</span> <span class="nx">soup.a</span>
<span class="k">link</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/elsie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link1"</span><span class="o">></span><span class="nx">Elsie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span>
<span class="k">link</span><span class="bp">.</span><span class="nx">next_sibling</span>
<span class="err">#</span> <span class="nx">u</span><span class="s1">',</span><span class="se">\n</span><span class="s1">'</span>
</pre></div>
<p>通过 <code>.next_siblings</code> 和 <code>.previous_siblings</code> 属性可以对当前节点的兄弟节点迭代输出:</p>
<div class="hlcode"><pre><span class="nb">for</span> <span class="n">sibling</span> <span class="k">in</span> <span class="nx">soup.a.next_siblings</span><span class="p">:</span>
<span class="nx">print</span><span class="p">(</span><span class="nx">repr</span><span class="p">(</span><span class="nx">sibling</span><span class="p">))</span>
<span class="err">#</span> <span class="nx">u</span><span class="s1">',</span><span class="se">\n</span><span class="s1">'</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/lacie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link2"</span><span class="o">></span><span class="nx">Lacie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span>
<span class="err">#</span> <span class="nx">u</span><span class="s1">' and</span><span class="se">\n</span><span class="s1">'</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/tillie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link3"</span><span class="o">></span><span class="nx">Tillie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span>
<span class="err">#</span> <span class="nx">u</span><span class="s1">'; and they lived at the bottom of a well.'</span>
<span class="err">#</span> <span class="kc">None</span>
</pre></div>
<h2 id="previous_sibling">.previous_sibling</h2>
<p>在文档树中,使用 <code>.next_sibling</code> 和 <code>.previous_sibling</code> 属性来查询兄弟节点:</p>
<div class="hlcode"><pre><span class="nx">sibling_soup.c.previous_sibling</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">b</span><span class="o">></span><span class="nx">text1</span><span class="o"></</span><span class="nx">b</span><span class="o">></span>
</pre></div>
<p>通过 <code>.next_siblings</code> 和 <code>.previous_siblings</code> 属性可以对当前节点的兄弟节点迭代输出:</p>
<div class="hlcode"><pre><span class="nb">for</span> <span class="n">sibling</span> <span class="k">in</span> <span class="nx">soup.find</span><span class="p">(</span><span class="n">id</span><span class="o">=</span><span class="s2">"link3"</span><span class="p">)</span><span class="bp">.</span><span class="nx">previous_siblings</span><span class="p">:</span>
<span class="nx">print</span><span class="p">(</span><span class="nx">repr</span><span class="p">(</span><span class="nx">sibling</span><span class="p">))</span>
<span class="err">#</span> <span class="s1">' and</span><span class="se">\n</span><span class="s1">'</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/lacie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link2"</span><span class="o">></span><span class="nx">Lacie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span>
<span class="err">#</span> <span class="nx">u</span><span class="s1">',</span><span class="se">\n</span><span class="s1">'</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/elsie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link1"</span><span class="o">></span><span class="nx">Elsie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span>
<span class="err">#</span> <span class="nx">u</span><span class="s1">'Once upon a time there were three little sisters; and their names were</span><span class="se">\n</span><span class="s1">'</span>
<span class="err">#</span> <span class="kc">None</span>
</pre></div>
<h2 id="next_element">.next_element</h2>
<p><code>.next_element</code> 属性指向解析过程中下一个被解析的对象(字符串或tag),结果可能与 <code>.next_sibling</code> 相同,但通常是不一样的.</p>
<div class="hlcode"><pre><span class="n">last_a_tag</span> <span class="o">=</span> <span class="nx">soup.find</span><span class="p">(</span><span class="s2">"a"</span><span class="p">,</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link3"</span><span class="p">)</span>
<span class="nx">last_a_tag</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/tillie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link3"</span><span class="o">></span><span class="nx">Tillie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span>
<span class="nx">last_a_tag.next_sibling</span>
<span class="err">#</span> <span class="s1">'; and they lived at the bottom of a well.'</span>
</pre></div>
<h2 id="previous_element">.previous_element</h2>
<p><code>.previous_element</code> 属性刚好与 <code>.next_element</code> 相反,它指向当前被解析的对象的前一个解析对象</p>
<div class="hlcode"><pre><span class="nx">last_a_tag.previous_element</span>
<span class="err">#</span> <span class="nx">u</span><span class="s1">' and</span><span class="se">\n</span><span class="s1">'</span>
<span class="nx">last_a_tag.previous_element.next_element</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/tillie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link3"</span><span class="o">></span><span class="nx">Tillie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span>
</pre></div>
<h2 id="next_elements">.next_elements</h2>
<p>通过 <code>.next_elements</code> 的迭代器就可以向前或向后访问文档的解析内容</p>
<div class="hlcode"><pre><span class="nb">for</span> <span class="n">element</span> <span class="k">in</span> <span class="nx">last_a_tag.next_elements</span><span class="p">:</span>
<span class="nx">print</span><span class="p">(</span><span class="nx">repr</span><span class="p">(</span><span class="nb">element</span><span class="p">))</span>
<span class="err">#</span> <span class="nx">u</span><span class="s1">'Tillie'</span>
<span class="err">#</span> <span class="nx">u</span><span class="s1">';</span><span class="se">\n</span><span class="s1">and they lived at the bottom of a well.'</span>
<span class="err">#</span> <span class="nx">u</span><span class="s1">'</span><span class="se">\n\n</span><span class="s1">'</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">p</span> <span class="n">class</span><span class="o">=</span><span class="s2">"story"</span><span class="o">></span><span class="nx">...</span><span class="o"></</span><span class="nx">p</span><span class="o">></span>
<span class="err">#</span> <span class="nx">u</span><span class="s1">'...'</span>
<span class="err">#</span> <span class="nx">u</span><span class="s1">'</span><span class="se">\n</span><span class="s1">'</span>
<span class="err">#</span> <span class="kc">None</span>
</pre></div>
<h2 id="previous_elements">.previous_elements</h2>
<p><code>.previous_elements</code> 就好像文档正在被解析一样</p>
<h1 id="html_1">搜索HTML</h1>
<h2 id="find">find</h2>
<p>find( <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#id32">name</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#css">attrs</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#recursive">recursive</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#text">text</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#keyword">**kwargs</a> )</p>
<p>找到第一个标签并返回，只会返回一个元素</p>
<p>使用 <code>find_all</code> 方法并设置 <code>limit=1</code> 参数不如直接使用 <code>find()</code> 方法.下面两行代码是等价的:</p>
<div class="hlcode"><pre><span class="nx">soup.find_all</span><span class="p">(</span><span class="s1">'title'</span><span class="p">,</span> <span class="n">limit</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
<span class="err">#</span> <span class="err">[</span><span class="o"><</span><span class="nb">title</span><span class="o">></span><span class="nx">The</span> <span class="nx">Dormouse</span><span class="s1">'s story</title>]</span>
<span class="s1">soup.find('</span><span class="nb">title</span><span class="s1">')</span>
<span class="s1"># <title>The Dormouse'</span><span class="nb">s</span> <span class="nx">story</span><span class="o"></</span><span class="nb">title</span><span class="o">></span>
</pre></div>
<p><code>find()</code> 方法找不到目标时,返回 <code>None</code> .</p>
<div class="hlcode"><pre><span class="n">print</span><span class="p">(</span><span class="n">soup</span><span class="p">.</span><span class="n">find</span><span class="p">(</span><span class="s">"nosuchtag"</span><span class="p">))</span>
<span class="cp"># None</span>
</pre></div>
<h2 id="find_all">find_all</h2>
<p>find_all( <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#id32">name</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#css">attrs</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#recursive">recursive</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#text">text</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#keyword">**kwargs</a> )</p>
<p><code>find_all()</code> 方法搜索当前tag的所有tag子节点,并判断是否符合过滤器的条件</p>
<p>找到所有的标签并返回，以列表形式</p>
<div class="hlcode"><pre><span class="n">In</span> <span class="p">[</span><span class="mi">5</span><span class="p">]</span><span class="o">:</span> <span class="n">import</span> <span class="n">requests</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">6</span><span class="p">]</span><span class="o">:</span> <span class="n">from</span> <span class="n">bs4</span> <span class="n">import</span> <span class="n">BeautifulSoup</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">7</span><span class="p">]</span><span class="o">:</span> <span class="n">url</span> <span class="o">=</span> <span class="err">'</span><span class="n">http</span><span class="o">:</span><span class="c1">//www.itest.info/courses'</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">8</span><span class="p">]</span><span class="o">:</span> <span class="n">soup</span> <span class="o">=</span> <span class="n">BeautifulSoup</span><span class="p">(</span><span class="n">requests</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">).</span><span class="n">text</span><span class="p">,</span> <span class="err">'</span><span class="n">html</span><span class="p">.</span><span class="n">parser</span><span class="err">'</span><span class="p">)</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">9</span><span class="p">]</span><span class="o">:</span> <span class="k">for</span> <span class="n">course</span> <span class="n">in</span> <span class="n">soup</span><span class="p">.</span><span class="n">find_all</span><span class="p">(</span><span class="err">'</span><span class="n">h4</span><span class="err">'</span><span class="p">)</span><span class="o">:</span>
<span class="p">...</span><span class="o">:</span> <span class="n">print</span><span class="p">(</span><span class="n">course</span><span class="p">.</span><span class="n">text</span><span class="p">)</span>
<span class="p">...</span><span class="o">:</span>
<span class="n">Android</span><span class="err">测试开发（</span><span class="n">Java</span><span class="err">语言）</span>
<span class="err">性能测试从入门到精通班</span>
<span class="err">接口自动化测试开发</span><span class="o">--</span><span class="n">Python</span><span class="err">班</span>
<span class="n">Selenium</span><span class="err">自动化测试</span><span class="o">--</span><span class="n">Java</span><span class="err">班</span>
<span class="n">Selenium</span><span class="err">自动化测试</span><span class="o">--</span><span class="n">Python</span><span class="err">班</span>
</pre></div>
<div class="hlcode"><pre><span class="nt"><span</span> <span class="na">class=</span><span class="s">"item_hot_topic_title"</span><span class="nt">></span>
<span class="nt"><a</span> <span class="na">href=</span><span class="s">"/t/415664"</span><span class="nt">></span>冬天了，身上静电咋处理？<span class="nt"></a></span>
<span class="nt"></span></span>
</pre></div>
<div class="hlcode"><pre><span class="k">for</span> <span class="n">span</span> <span class="n">in</span> <span class="n">soup</span><span class="p">.</span><span class="n">find_all</span><span class="p">(</span><span class="err">'</span><span class="n">span</span><span class="err">'</span><span class="p">,</span> <span class="n">class_</span><span class="o">=</span><span class="err">'</span><span class="n">item_hot_topic_title</span><span class="err">'</span><span class="p">)</span><span class="o">:</span>
<span class="n">print</span><span class="p">(</span><span class="n">span</span><span class="p">.</span><span class="n">find</span><span class="p">(</span><span class="sc">'a'</span><span class="p">).</span><span class="n">text</span><span class="p">,</span> <span class="n">span</span><span class="p">.</span><span class="n">find</span><span class="p">(</span><span class="sc">'a'</span><span class="p">)[</span><span class="err">'</span><span class="n">href</span><span class="err">'</span><span class="p">])</span>
</pre></div>
<blockquote>
<p><code>for span in soup.find_all('span', class_='item_hot_topic_title')</code>: 遍历所有的class=item_hot_topic_title的span。注意是<code>class_</code>，不是<code>class</code>，因为class是python的关键字，所以后面要加个尾巴，防止冲突</p>
</blockquote>
<h3 id="attrs">attrs</h3>
<p>通过attrs参数将属性条件返回到字典中</p>
<div class="hlcode"><pre><span class="nx">data_soup.find_all</span><span class="p">(</span><span class="n">attrs</span><span class="o">=</span><span class="p">{</span><span class="s2">"data-foo"</span><span class="p">:</span> <span class="s2">"value"</span><span class="p">})</span>
<span class="err">#</span> <span class="err">[</span><span class="o"><</span><span class="nb">div</span> <span class="kd">data</span><span class="na">-foo</span><span class="o">=</span><span class="s2">"value"</span><span class="o">></span><span class="nb">foo</span><span class="o">!</</span><span class="nb">div</span><span class="o">></span><span class="cp">]</span>
</pre></div>
<div class="hlcode"><pre><span class="nx">soup.find_all</span><span class="p">(</span><span class="s2">"a"</span><span class="p">,</span> <span class="n">attrs</span><span class="o">=</span><span class="p">{</span><span class="s2">"class"</span><span class="p">:</span> <span class="s2">"sister"</span><span class="p">})</span>
<span class="err">#</span> <span class="err">[</span><span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/elsie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link1"</span><span class="o">></span><span class="nx">Elsie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="p">,</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/lacie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link2"</span><span class="o">></span><span class="nx">Lacie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="p">,</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/tillie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link3"</span><span class="o">></span><span class="nx">Tillie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="cp">]</span>
</pre></div>
<h3 id="_5">列表搜索</h3>
<p>如果传入列表参数,Beautiful Soup会将与列表中任一元素匹配的内容返回.下面代码找到文档中所有<a>标签和<b>标签:</p>
<div class="hlcode"><pre><span class="nx">soup.find_all</span><span class="p">(</span><span class="err">[</span><span class="s2">"a"</span><span class="p">,</span> <span class="s2">"b"</span><span class="cp">]</span>)
# <span class="cp">[</span><span class="o"><</span><span class="nx">b</span><span class="o">></span><span class="nx">The</span> <span class="nx">Dormouse</span><span class="s1">'s story</b>,</span>
<span class="s1"># <a class="sister" href="http://example.com/elsie" id="link1">Elsie</a>,</span>
<span class="s1"># <a class="sister" href="http://example.com/lacie" id="link2">Lacie</a>,</span>
<span class="s1"># <a class="sister" href="http://example.com/tillie" id="link3">Tillie</a>]</span>
</pre></div>
<h3 id="true">True</h3>
<p><code>True</code> 可以匹配任何值,下面代码查找到所有的tag,但是不会返回字符串节点</p>
<div class="hlcode"><pre><span class="k">for</span> <span class="n">tag</span> <span class="n">in</span> <span class="n">soup</span><span class="p">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">True</span><span class="p">)</span><span class="o">:</span>
<span class="n">print</span><span class="p">(</span><span class="n">tag</span><span class="p">.</span><span class="n">name</span><span class="p">)</span>
<span class="cp"># html</span>
<span class="cp"># head</span>
<span class="cp"># title</span>
<span class="cp"># body</span>
<span class="cp"># p</span>
<span class="cp"># b</span>
<span class="cp"># p</span>
<span class="cp"># a</span>
<span class="cp"># a</span>
<span class="cp"># a</span>
<span class="cp"># p</span>
</pre></div>
<h3 id="css">CSS搜索</h3>
<p>从Beautiful Soup的4.1.1版本开始,可以通过 <code>class_</code> 参数搜索有指定CSS类名的tag</p>
<div class="hlcode"><pre><span class="nx">soup.find_all</span><span class="p">(</span><span class="s2">"a"</span><span class="p">,</span> <span class="n">class_</span><span class="o">=</span><span class="s2">"sister"</span><span class="p">)</span>
<span class="err">#</span> <span class="err">[</span><span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/elsie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link1"</span><span class="o">></span><span class="nx">Elsie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="p">,</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/lacie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link2"</span><span class="o">></span><span class="nx">Lacie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="p">,</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/tillie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link3"</span><span class="o">></span><span class="nx">Tillie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="cp">]</span>
</pre></div>
<h3 id="text">text 参数</h3>
<p>通过 <code>text</code> 参数可以搜搜文档中的字符串内容.与 <code>name</code> 参数的可选值一样, <code>text</code> 参数接受 <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#id27">字符串</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#id28">正则表达式</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#id29">列表</a>, True</p>
<div class="hlcode"><pre><span class="n">soup</span><span class="p">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">text</span><span class="o">=</span><span class="s">"Elsie"</span><span class="p">)</span>
<span class="cp"># [u'Elsie']</span>
<span class="n">soup</span><span class="p">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">text</span><span class="o">=</span><span class="p">[</span><span class="s">"Tillie"</span><span class="p">,</span> <span class="s">"Elsie"</span><span class="p">,</span> <span class="s">"Lacie"</span><span class="p">])</span>
<span class="cp"># [u'Elsie', u'Lacie', u'Tillie']</span>
<span class="n">soup</span><span class="p">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">text</span><span class="o">=</span><span class="n">re</span><span class="p">.</span><span class="n">compile</span><span class="p">(</span><span class="s">"Dormouse"</span><span class="p">))</span>
<span class="p">[</span><span class="n">u</span><span class="s">"The Dormouse's story"</span><span class="p">,</span> <span class="n">u</span><span class="s">"The Dormouse's story"</span><span class="p">]</span>
<span class="n">def</span> <span class="n">is_the_only_string_within_a_tag</span><span class="p">(</span><span class="n">s</span><span class="p">)</span><span class="o">:</span>
<span class="s">""</span><span class="n">Return</span> <span class="n">True</span> <span class="k">if</span> <span class="n">this</span> <span class="n">string</span> <span class="n">is</span> <span class="n">the</span> <span class="n">only</span> <span class="n">child</span> <span class="n">of</span> <span class="n">its</span> <span class="n">parent</span> <span class="n">tag</span><span class="p">.</span><span class="s">""</span>
<span class="k">return</span> <span class="p">(</span><span class="n">s</span> <span class="o">==</span> <span class="n">s</span><span class="p">.</span><span class="n">parent</span><span class="p">.</span><span class="n">string</span><span class="p">)</span>
<span class="n">soup</span><span class="p">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">text</span><span class="o">=</span><span class="n">is_the_only_string_within_a_tag</span><span class="p">)</span>
<span class="cp"># [u"The Dormouse's story", u"The Dormouse's story", u'Elsie', u'Lacie', u'Tillie', u'...']</span>
</pre></div>
<div class="hlcode"><pre><span class="nx">soup.find_all</span><span class="p">(</span><span class="s2">"a"</span><span class="p">,</span> <span class="n">text</span><span class="o">=</span><span class="s2">"Elsie"</span><span class="p">)</span>
<span class="err">#</span> <span class="err">[</span><span class="o"><</span><span class="nx">a</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/elsie"</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link1"</span><span class="o">></span><span class="nx">Elsie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="cp">]</span>
</pre></div>
<h3 id="limit">limit 参数</h3>
<p><code>find_all()</code> 方法返回全部的搜索结构,如果文档树很大那么搜索会很慢.如果我们不需要全部结果,可以使用 <code>limit</code> 参数限制返回结果的数量.效果与SQL中的limit关键字类似,当搜索到的结果数量达到 <code>limit</code> 的限制时,就停止搜索返回结果.</p>
<p>文档树中有3个tag符合搜索条件,但结果只返回了2个,因为我们限制了返回数量:</p>
<div class="hlcode"><pre><span class="nx">soup.find_all</span><span class="p">(</span><span class="s2">"a"</span><span class="p">,</span> <span class="n">limit</span><span class="o">=</span><span class="mi">2</span><span class="p">)</span>
<span class="err">#</span> <span class="err">[</span><span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/elsie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link1"</span><span class="o">></span><span class="nx">Elsie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="p">,</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/lacie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link2"</span><span class="o">></span><span class="nx">Lacie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="cp">]</span>
</pre></div>
<h3 id="recursive">recursive 参数</h3>
<p>调用tag的 <code>find_all()</code> 方法时,Beautiful Soup会检索当前tag的所有子孙节点,如果只想搜索tag的直接子节点,可以使用参数 <code>recursive=False</code> .</p>
<p>一段简单的文档:</p>
<div class="hlcode"><pre><span class="nt"><html></span>
<span class="nt"><head></span>
<span class="nt"><title></span>
The Dormouse's story
<span class="nt"></title></span>
<span class="nt"></head></span>
...
</pre></div>
<p>是否使用 <code>recursive</code> 参数的搜索结果:</p>
<div class="hlcode"><pre><span class="nx">soup.html.find_all</span><span class="p">(</span><span class="s2">"title"</span><span class="p">)</span>
<span class="err">#</span> <span class="err">[</span><span class="o"><</span><span class="nb">title</span><span class="o">></span><span class="nx">The</span> <span class="nx">Dormouse</span><span class="s1">'s story</title>]</span>
<span class="s1">soup.html.find_all("title", recursive=False)</span>
<span class="s1"># []</span>
</pre></div>
<h3 id="_6">正则表达式</h3>
<p>如果传入正则表达式作为参数,Beautiful Soup会通过正则表达式的 <code>match()</code> 来匹配内容.下面例子中找出所有以b开头的标签,这表示<body>和<b>标签都应该被找到:</p>
<div class="hlcode"><pre><span class="n">import</span> <span class="n">re</span>
<span class="k">for</span> <span class="n">tag</span> <span class="n">in</span> <span class="n">soup</span><span class="p">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">re</span><span class="p">.</span><span class="n">compile</span><span class="p">(</span><span class="s">"^b"</span><span class="p">))</span><span class="o">:</span>
<span class="n">print</span><span class="p">(</span><span class="n">tag</span><span class="p">.</span><span class="n">name</span><span class="p">)</span>
<span class="cp"># body</span>
<span class="cp"># b</span>
</pre></div>
<p>下面代码找出所有名字中包含”t”的标签:</p>
<div class="hlcode"><pre><span class="k">for</span> <span class="n">tag</span> <span class="n">in</span> <span class="n">soup</span><span class="p">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">re</span><span class="p">.</span><span class="n">compile</span><span class="p">(</span><span class="s">"t"</span><span class="p">))</span><span class="o">:</span>
<span class="n">print</span><span class="p">(</span><span class="n">tag</span><span class="p">.</span><span class="n">name</span><span class="p">)</span>
<span class="cp"># html</span>
<span class="cp"># title</span>
</pre></div>
<h2 id="find_parents-find_parent">find_parents() 和 find_parent()</h2>
<p>find_parents( <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#id32">name</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#css">attrs</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#recursive">recursive</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#text">text</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#keyword">**kwargs</a> )</p>
<p>find_parent( <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#id32">name</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#css">attrs</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#recursive">recursive</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#text">text</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#keyword">**kwargs</a> )</p>
<p><code>find_parents()</code> 和 <code>find_parent()</code> 用来搜索当前节点的父辈节点,搜索方法与普通tag的搜索方法相同,搜索文档搜索文档包含的内容.</p>
<div class="hlcode"><pre><span class="n">a_string</span> <span class="o">=</span> <span class="nx">soup.find</span><span class="p">(</span><span class="n">text</span><span class="o">=</span><span class="s2">"Lacie"</span><span class="p">)</span>
<span class="nx">a_string</span>
<span class="err">#</span> <span class="nx">u</span><span class="s1">'Lacie'</span>
<span class="nx">a_string.find_parents</span><span class="p">(</span><span class="s2">"a"</span><span class="p">)</span>
<span class="err">#</span> <span class="err">[</span><span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/lacie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link2"</span><span class="o">></span><span class="nx">Lacie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="cp">]</span>
a_string.find_parent("p")
# <span class="nt"><p</span> <span class="na">class=</span><span class="s">"story"</span><span class="nt">></span>Once upon a time there were three little sisters; and their names were
# <span class="nt"><a</span> <span class="na">class=</span><span class="s">"sister"</span> <span class="na">href=</span><span class="s">"http://example.com/elsie"</span> <span class="na">id=</span><span class="s">"link1"</span><span class="nt">></span>Elsie<span class="nt"></a></span>,
# <span class="nt"><a</span> <span class="na">class=</span><span class="s">"sister"</span> <span class="na">href=</span><span class="s">"http://example.com/lacie"</span> <span class="na">id=</span><span class="s">"link2"</span><span class="nt">></span>Lacie<span class="nt"></a></span> and
# <span class="nt"><a</span> <span class="na">class=</span><span class="s">"sister"</span> <span class="na">href=</span><span class="s">"http://example.com/tillie"</span> <span class="na">id=</span><span class="s">"link3"</span><span class="nt">></span>Tillie<span class="nt"></a></span>;
# and they lived at the bottom of a well.<span class="nt"></p></span>
a_string.find_parents("p", class="title")
# <span class="cp">[]</span>
</pre></div>
<h2 id="find_next_siblings-find_next_sibling">find_next_siblings() 合 find_next_sibling()</h2>
<p>find_next_siblings( <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#id32">name</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#css">attrs</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#recursive">recursive</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#text">text</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#keyword">**kwargs</a> )</p>
<p>find_next_sibling( <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#id32">name</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#css">attrs</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#recursive">recursive</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#text">text</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#keyword">**kwargs</a> )</p>
<p><code>find_next_siblings()</code> 方法返回所有符合条件的后面的兄弟节点, <code>find_next_sibling()</code> 只返回符合条件的后面的第一个tag节点.</p>
<div class="hlcode"><pre><span class="n">first_link</span> <span class="o">=</span> <span class="nx">soup.a</span>
<span class="nx">first_link</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/elsie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link1"</span><span class="o">></span><span class="nx">Elsie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span>
<span class="nx">first_link.find_next_siblings</span><span class="p">(</span><span class="s2">"a"</span><span class="p">)</span>
<span class="err">#</span> <span class="err">[</span><span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/lacie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link2"</span><span class="o">></span><span class="nx">Lacie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="p">,</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/tillie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link3"</span><span class="o">></span><span class="nx">Tillie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="cp">]</span>
first_story_paragraph = soup.find("p", "story")
first_story_paragraph.find_next_sibling("p")
# <span class="nt"><p</span> <span class="na">class=</span><span class="s">"story"</span><span class="nt">></span>...<span class="nt"></p></span>
</pre></div>
<h2 id="find_previous_siblings-find_previous_sibling">find_previous_siblings() 和 find_previous_sibling()</h2>
<p>find_previous_siblings( <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#id32">name</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#css">attrs</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#recursive">recursive</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#text">text</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#keyword">**kwargs</a> )</p>
<p>find_previous_sibling( <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#id32">name</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#css">attrs</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#recursive">recursive</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#text">text</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#keyword">**kwargs</a> )</p>
<p><code>find_previous_siblings()</code> 方法返回所有符合条件的前面的兄弟节点, <code>find_previous_sibling()</code> 方法返回第一个符合条件的前面的兄弟节点:</p>
<div class="hlcode"><pre><span class="n">last_link</span> <span class="o">=</span> <span class="nx">soup.find</span><span class="p">(</span><span class="s2">"a"</span><span class="p">,</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link3"</span><span class="p">)</span>
<span class="nx">last_link</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/tillie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link3"</span><span class="o">></span><span class="nx">Tillie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span>
<span class="nx">last_link.find_previous_siblings</span><span class="p">(</span><span class="s2">"a"</span><span class="p">)</span>
<span class="err">#</span> <span class="err">[</span><span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/lacie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link2"</span><span class="o">></span><span class="nx">Lacie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="p">,</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/elsie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link1"</span><span class="o">></span><span class="nx">Elsie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="cp">]</span>
first_story_paragraph = soup.find("p", "story")
first_story_paragraph.find_previous_sibling("p")
# <span class="nt"><p</span> <span class="na">class=</span><span class="s">"title"</span><span class="nt">><b></span>The Dormouse's story<span class="nt"></b></p></span>
</pre></div>
<h2 id="find_all_next-find_next">find_all_next() 和 find_next()</h2>
<p>find_all_next( <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#id32">name</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#css">attrs</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#recursive">recursive</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#text">text</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#keyword">**kwargs</a> )</p>
<p>find_next( <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#id32">name</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#css">attrs</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#recursive">recursive</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#text">text</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#keyword">**kwargs</a> )</p>
<p><code>find_all_next()</code> 方法返回所有符合条件的节点, <code>find_next()</code> 方法返回第一个符合条件的节点:</p>
<div class="hlcode"><pre><span class="n">first_link</span> <span class="o">=</span> <span class="nx">soup.a</span>
<span class="nx">first_link</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/elsie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link1"</span><span class="o">></span><span class="nx">Elsie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span>
<span class="nx">first_link.find_all_next</span><span class="p">(</span><span class="n">text</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="err">#</span> <span class="err">[</span><span class="nx">u</span><span class="s1">'Elsie'</span><span class="p">,</span> <span class="nx">u</span><span class="s1">',</span><span class="se">\n</span><span class="s1">'</span><span class="p">,</span> <span class="nx">u</span><span class="s1">'Lacie'</span><span class="p">,</span> <span class="nx">u</span><span class="s1">' and</span><span class="se">\n</span><span class="s1">'</span><span class="p">,</span> <span class="nx">u</span><span class="s1">'Tillie'</span><span class="p">,</span>
<span class="err">#</span> <span class="nx">u</span><span class="s1">';</span><span class="se">\n</span><span class="s1">and they lived at the bottom of a well.'</span><span class="p">,</span> <span class="nx">u</span><span class="s1">'</span><span class="se">\n\n</span><span class="s1">'</span><span class="p">,</span> <span class="nx">u</span><span class="s1">'...'</span><span class="p">,</span> <span class="nx">u</span><span class="s1">'</span><span class="se">\n</span><span class="s1">'</span><span class="cp">]</span>
first_link.find_next("p")
# <span class="nt"><p</span> <span class="na">class=</span><span class="s">"story"</span><span class="nt">></span>...<span class="nt"></p></span>
</pre></div>
<h2 id="find_all_previous-find_previous">find_all_previous() 和 find_previous()</h2>
<p>find_all_previous( <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#id32">name</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#css">attrs</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#recursive">recursive</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#text">text</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#keyword">**kwargs</a> )</p>
<p>find_previous( <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#id32">name</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#css">attrs</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#recursive">recursive</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#text">text</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#keyword">**kwargs</a> )</p>
<p><code>find_all_previous()</code> 方法返回所有符合条件的节点, <code>find_previous()</code> 方法返回第一个符合条件的节点.</p>
<div class="hlcode"><pre><span class="n">first_link</span> <span class="o">=</span> <span class="nx">soup.a</span>
<span class="nx">first_link</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/elsie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link1"</span><span class="o">></span><span class="nx">Elsie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span>
<span class="nx">first_link.find_all_previous</span><span class="p">(</span><span class="s2">"p"</span><span class="p">)</span>
<span class="err">#</span> <span class="err">[</span><span class="o"><</span><span class="nx">p</span> <span class="n">class</span><span class="o">=</span><span class="s2">"story"</span><span class="o">></span><span class="nb">Once</span> <span class="nx">upon</span> <span class="nx">a</span> <span class="nb">time</span> <span class="nx">there</span> <span class="nx">were</span> <span class="nx">three</span> <span class="nx">little</span> <span class="nx">sisters</span><span class="p">;</span> <span class="nx">...</span><span class="o"></</span><span class="nx">p</span><span class="o">></span><span class="p">,</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">p</span> <span class="n">class</span><span class="o">=</span><span class="s2">"title"</span><span class="o">><</span><span class="nx">b</span><span class="o">></span><span class="nx">The</span> <span class="nx">Dormouse</span><span class="s1">'s story</b></p>]</span>
<span class="s1">first_link.find_previous("title")</span>
<span class="s1"># <title>The Dormouse'</span><span class="nb">s</span> <span class="nx">story</span><span class="o"></</span><span class="nb">title</span><span class="o">></span>
</pre></div>
<h2 id="css-select">CSS选择器 select 方法</h2>
<h3 id="_7">通过标签名称查找</h3>
<p><code>select()</code> 方法中传入字符串参数,即可使用CSS选择器的语法找到tag</p>
<div class="hlcode"><pre><span class="nx">soup.select</span><span class="p">(</span><span class="s2">"title"</span><span class="p">)</span>
<span class="err">#</span> <span class="err">[</span><span class="o"><</span><span class="nb">title</span><span class="o">></span><span class="nx">The</span> <span class="nx">Dormouse</span><span class="s1">'s story</title>]</span>
<span class="s1">soup.select("p nth-of-type(3)")</span>
<span class="s1"># [<p class="story">...</p>]</span>
</pre></div>
<p>通过tag标签逐层查找</p>
<div class="hlcode"><pre><span class="nx">soup.select</span><span class="p">(</span><span class="s2">"body a"</span><span class="p">)</span>
<span class="err">#</span> <span class="err">[</span><span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/elsie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link1"</span><span class="o">></span><span class="nx">Elsie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="p">,</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/lacie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link2"</span><span class="o">></span><span class="nx">Lacie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="p">,</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/tillie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link3"</span><span class="o">></span><span class="nx">Tillie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="cp">]</span>
soup.select("html head title")
# <span class="cp">[</span><span class="o"><</span><span class="nb">title</span><span class="o">></span><span class="nx">The</span> <span class="nx">Dormouse</span><span class="s1">'s story</title>]</span>
</pre></div>
<p>找到某个tag标签下的直接子标签</p>
<div class="hlcode"><pre><span class="nx">soup.select</span><span class="p">(</span><span class="s2">"head > title"</span><span class="p">)</span>
<span class="err">#</span> <span class="err">[</span><span class="o"><</span><span class="nb">title</span><span class="o">></span><span class="nx">The</span> <span class="nx">Dormouse</span><span class="s1">'s story</title>]</span>
<span class="s1">soup.select("p > a")</span>
<span class="s1"># [<a class="sister" href="http://example.com/elsie" id="link1">Elsie</a>,</span>
<span class="s1"># <a class="sister" href="http://example.com/lacie" id="link2">Lacie</a>,</span>
<span class="s1"># <a class="sister" href="http://example.com/tillie" id="link3">Tillie</a>]</span>
<span class="s1">soup.select("p > a:nth-of-type(2)")</span>
<span class="s1"># [<a class="sister" href="http://example.com/lacie" id="link2">Lacie</a>]</span>
<span class="s1">soup.select("p > #link1")</span>
<span class="s1"># [<a class="sister" href="http://example.com/elsie" id="link1">Elsie</a>]</span>
<span class="s1">soup.select("body > a")</span>
<span class="s1"># []</span>
</pre></div>
<p>找到兄弟节点标签:</p>
<div class="hlcode"><pre><span class="nx">soup.select</span><span class="p">(</span><span class="s2">"#link1 ~ .sister"</span><span class="p">)</span>
<span class="err">#</span> <span class="err">[</span><span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/lacie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link2"</span><span class="o">></span><span class="nx">Lacie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="p">,</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/tillie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link3"</span><span class="o">></span><span class="nx">Tillie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="cp">]</span>
soup.select("#link1 + .sister")
# <span class="cp">[</span><span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/lacie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link2"</span><span class="o">></span><span class="nx">Lacie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="cp">]</span>
</pre></div>
<h3 id="css_1">通过CSS的类名查找</h3>
<div class="hlcode"><pre><span class="nx">soup.select</span><span class="p">(</span><span class="s2">".sister"</span><span class="p">)</span>
<span class="err">#</span> <span class="err">[</span><span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/elsie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link1"</span><span class="o">></span><span class="nx">Elsie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="p">,</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/lacie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link2"</span><span class="o">></span><span class="nx">Lacie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="p">,</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/tillie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link3"</span><span class="o">></span><span class="nx">Tillie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="cp">]</span>
soup.select("<span class="cp">[</span><span class="nb">class</span><span class="err">~</span><span class="o">=</span><span class="nx">sister</span><span class="cp">]</span>")
# <span class="cp">[</span><span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/elsie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link1"</span><span class="o">></span><span class="nx">Elsie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="p">,</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/lacie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link2"</span><span class="o">></span><span class="nx">Lacie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="p">,</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/tillie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link3"</span><span class="o">></span><span class="nx">Tillie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="cp">]</span>
</pre></div>
<h3 id="tagid">通过tag的id查找</h3>
<div class="hlcode"><pre><span class="nx">soup.select</span><span class="p">(</span><span class="s2">"#link1"</span><span class="p">)</span>
<span class="err">#</span> <span class="err">[</span><span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/elsie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link1"</span><span class="o">></span><span class="nx">Elsie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="cp">]</span>
soup.select("a#link2")
# <span class="cp">[</span><span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/lacie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link2"</span><span class="o">></span><span class="nx">Lacie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="cp">]</span>
</pre></div>
<h3 id="_8">组合查找</h3>
<div class="hlcode"><pre><span class="n">soup</span><span class="p">.</span><span class="n">select</span><span class="p">(</span><span class="err">'</span><span class="n">p</span> <span class="err">#</span><span class="n">link1</span><span class="err">'</span><span class="p">)</span>
</pre></div>
<div class="hlcode"><pre><span class="nt"><div</span> <span class="na">class=</span><span class="s">"line"</span><span class="nt">></span> div tag <span class="nt"></div></span>
soup.select('div.line')
</pre></div>
<h3 id="_9">通过是否存在某个属性来查找</h3>
<div class="hlcode"><pre><span class="nx">soup.select</span><span class="p">(</span><span class="s1">'a[href]'</span><span class="p">)</span>
<span class="err">#</span> <span class="err">[</span><span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/elsie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link1"</span><span class="o">></span><span class="nx">Elsie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="p">,</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/lacie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link2"</span><span class="o">></span><span class="nx">Lacie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="p">,</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/tillie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link3"</span><span class="o">></span><span class="nx">Tillie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="cp">]</span>
</pre></div>
<h3 id="_10">通过属性的值来查找</h3>
<div class="hlcode"><pre><span class="nx">soup.select</span><span class="p">(</span><span class="s1">'a[href="http://example.com/elsie"]'</span><span class="p">)</span>
<span class="err">#</span> <span class="err">[</span><span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/elsie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link1"</span><span class="o">></span><span class="nx">Elsie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="cp">]</span>
soup.select('a<span class="cp">[</span><span class="nb">href</span><span class="p">^</span><span class="o">=</span><span class="s2">"http://example.com/"</span><span class="cp">]</span>')
# <span class="cp">[</span><span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/elsie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link1"</span><span class="o">></span><span class="nx">Elsie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="p">,</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/lacie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link2"</span><span class="o">></span><span class="nx">Lacie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="p">,</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/tillie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link3"</span><span class="o">></span><span class="nx">Tillie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="cp">]</span>
soup.select('a<span class="cp">[</span><span class="nb">href</span><span class="err">$</span><span class="o">=</span><span class="s2">"tillie"</span><span class="cp">]</span>')
# <span class="cp">[</span><span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/tillie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link3"</span><span class="o">></span><span class="nx">Tillie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="cp">]</span>
soup.select('a<span class="cp">[</span><span class="nb">href</span><span class="o">*=</span><span class="s2">".com/el"</span><span class="cp">]</span>')
# <span class="cp">[</span><span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/elsie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link1"</span><span class="o">></span><span class="nx">Elsie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="cp">]</span>
</pre></div>
<h3 id="_11">通过语言设置来查找</h3>
<div class="hlcode"><pre><span class="n">multilingual_markup</span> <span class="o">=</span> <span class="s2">"""</span>
<span class="s2"> <p lang="</span><span class="nx">en</span><span class="s2">">Hello</p></span>
<span class="s2"> <p lang="</span><span class="nx">en</span><span class="na">-us</span><span class="s2">">Howdy, y'all</p></span>
<span class="s2"> <p lang="</span><span class="nx">en</span><span class="na">-gb</span><span class="s2">">Pip-pip, old fruit</p></span>
<span class="s2"> <p lang="</span><span class="nx">fr</span><span class="s2">">Bonjour mes amis</p></span>
<span class="s2">"""</span>
<span class="n">multilingual_soup</span> <span class="o">=</span> <span class="nx">BeautifulSoup</span><span class="p">(</span><span class="nx">multilingual_markup</span><span class="p">)</span>
<span class="nx">multilingual_soup.select</span><span class="p">(</span><span class="s1">'p[lang|=en]'</span><span class="p">)</span>
<span class="err">#</span> <span class="err">[</span><span class="o"><</span><span class="nx">p</span> <span class="n">lang</span><span class="o">=</span><span class="s2">"en"</span><span class="o">></span><span class="nx">Hello</span><span class="o"></</span><span class="nx">p</span><span class="o">></span><span class="p">,</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">p</span> <span class="n">lang</span><span class="o">=</span><span class="s2">"en-us"</span><span class="o">></span><span class="nx">Howdy</span><span class="p">,</span> <span class="nx">y</span><span class="s1">'all</p>,</span>
<span class="s1"># <p lang="en-gb">Pip-pip, old fruit</p>]</span>
</pre></div>
<h1 id="html_2">修改HTML</h1>
<p>Beautiful Soup的强项是文档树的搜索,但同时也可以方便的修改文档树</p>
<h2 id="tag_2">修改tag的名称和属性</h2>
<p>在 <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#attributes">Attributes</a> 的章节中已经介绍过这个功能,但是再看一遍也无妨. 重命名一个tag,改变属性的值,添加或删除属性:</p>
<div class="hlcode"><pre><span class="n">soup</span> <span class="o">=</span> <span class="nx">BeautifulSoup</span><span class="p">(</span><span class="s1">'<b class="boldest">Extremely bold</b>'</span><span class="p">)</span>
<span class="kt">tag</span> <span class="o">=</span> <span class="nx">soup.b</span>
<span class="kt">tag</span><span class="bp">.</span><span class="nb">name</span> <span class="o">=</span> <span class="s2">"blockquote"</span>
<span class="kt">tag</span><span class="err">[</span><span class="s1">'class'</span><span class="cp">]</span> = 'verybold'
tag<span class="cp">[</span><span class="s1">'id'</span><span class="cp">]</span> = 1
tag
# <span class="nt"><blockquote</span> <span class="na">class=</span><span class="s">"verybold"</span> <span class="na">id=</span><span class="s">"1"</span><span class="nt">></span>Extremely bold<span class="nt"></blockquote></span>
del tag<span class="cp">[</span><span class="s1">'class'</span><span class="cp">]</span>
del tag<span class="cp">[</span><span class="s1">'id'</span><span class="cp">]</span>
tag
# <span class="nt"><blockquote></span>Extremely bold<span class="nt"></blockquote></span>
</pre></div>
<h2 id="string_1">修改 .string</h2>
<p>给tag的 <code>.string</code> 属性赋值,就相当于用当前的内容替代了原来的内容:</p>
<div class="hlcode"><pre><span class="n">markup</span> <span class="o">=</span> <span class="s1">'<a href="http://example.com/">I linked to <i>example.com</i></a>'</span>
<span class="n">soup</span> <span class="o">=</span> <span class="nx">BeautifulSoup</span><span class="p">(</span><span class="nx">markup</span><span class="p">)</span>
<span class="kt">tag</span> <span class="o">=</span> <span class="nx">soup.a</span>
<span class="kt">tag</span><span class="bp">.</span><span class="nb">string</span> <span class="o">=</span> <span class="s2">"New link text."</span>
<span class="kt">tag</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/"</span><span class="o">></span><span class="nb">New</span> <span class="k">link</span> <span class="nx">text.</span><span class="o"></</span><span class="nx">a</span><span class="o">></span>
</pre></div>
<blockquote>
<p>如果当前的tag包含了其它tag,那么给它的 <code>.string</code> 属性赋值会覆盖掉原有的所有内容包括子tag</p>
</blockquote>
<h2 id="append">append()</h2>
<p><code>Tag.append()</code> 方法想tag中添加内容,就好像Python的列表的 <code>.append()</code> 方法:</p>
<div class="hlcode"><pre><span class="n">soup</span> <span class="o">=</span> <span class="nx">BeautifulSoup</span><span class="p">(</span><span class="s2">"<a>Foo</a>"</span><span class="p">)</span>
<span class="nx">soup.a.append</span><span class="p">(</span><span class="s2">"Bar"</span><span class="p">)</span>
<span class="nx">soup</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">html</span><span class="o">><</span><span class="nb">head</span><span class="o">></</span><span class="nb">head</span><span class="o">><</span><span class="nb">body</span><span class="o">><</span><span class="nx">a</span><span class="o">></span><span class="nx">FooBar</span><span class="o"></</span><span class="nx">a</span><span class="o">></</span><span class="nb">body</span><span class="o">></</span><span class="nx">html</span><span class="o">></span>
<span class="nx">soup.a.contents</span>
<span class="err">#</span> <span class="err">[</span><span class="nx">u</span><span class="s1">'Foo'</span><span class="p">,</span> <span class="nx">u</span><span class="s1">'Bar'</span><span class="cp">]</span>
</pre></div>
<h2 id="beautifulsoupnew_string-new_tag">BeautifulSoup.new_string() 和 .new_tag()</h2>
<p>如果想添加一段文本内容到文档中也没问题,可以调用Python的 <code>append()</code> 方法或调用工厂方法 <code>BeautifulSoup.new_string()</code> :</p>
<div class="hlcode"><pre><span class="n">soup</span> <span class="o">=</span> <span class="nx">BeautifulSoup</span><span class="p">(</span><span class="s2">"<b></b>"</span><span class="p">)</span>
<span class="kt">tag</span> <span class="o">=</span> <span class="nx">soup.b</span>
<span class="kt">tag</span><span class="bp">.</span><span class="nb">append</span><span class="p">(</span><span class="s2">"Hello"</span><span class="p">)</span>
<span class="n">new_string</span> <span class="o">=</span> <span class="nx">soup.new_string</span><span class="p">(</span><span class="s2">" there"</span><span class="p">)</span>
<span class="kt">tag</span><span class="bp">.</span><span class="nb">append</span><span class="p">(</span><span class="nx">new_string</span><span class="p">)</span>
<span class="kt">tag</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">b</span><span class="o">></span><span class="nx">Hello</span> <span class="nx">there.</span><span class="o"></</span><span class="nx">b</span><span class="o">></span>
<span class="kt">tag</span><span class="bp">.</span><span class="nb">contents</span>
<span class="err">#</span> <span class="err">[</span><span class="nx">u</span><span class="s1">'Hello'</span><span class="p">,</span> <span class="nx">u</span><span class="s1">' there'</span><span class="cp">]</span>
</pre></div>
<p>如果想要创建一段注释,或 <code>NavigableString</code> 的任何子类,将子类作为 <code>new_string()</code> 方法的第二个参数传入:</p>
<div class="hlcode"><pre><span class="nb">from</span> <span class="nx">bs4</span> <span class="k">import</span> <span class="nx">Comment</span>
<span class="n">new_comment</span> <span class="o">=</span> <span class="nx">soup.new_string</span><span class="p">(</span><span class="s2">"Nice to see you."</span><span class="p">,</span> <span class="nx">Comment</span><span class="p">)</span>
<span class="kt">tag</span><span class="bp">.</span><span class="nb">append</span><span class="p">(</span><span class="nx">new_comment</span><span class="p">)</span>
<span class="kt">tag</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">b</span><span class="o">></span><span class="nx">Hello</span> <span class="nx">there</span><span class="o"><!--</span><span class="nx">Nice</span> <span class="k">to</span> <span class="nx">see</span> <span class="nx">you.</span><span class="o">--></</span><span class="nx">b</span><span class="o">></span>
<span class="kt">tag</span><span class="bp">.</span><span class="nb">contents</span>
<span class="err">#</span> <span class="err">[</span><span class="nx">u</span><span class="s1">'Hello'</span><span class="p">,</span> <span class="nx">u</span><span class="s1">' there'</span><span class="p">,</span> <span class="nx">u</span><span class="s1">'Nice to see you.'</span><span class="cp">]</span>
</pre></div>
<p># 这是Beautiful Soup 4.2.1 中新增的方法</p>
<p>创建一个tag最好的方法是调用工厂方法 <code>BeautifulSoup.new_tag()</code> :</p>
<div class="hlcode"><pre><span class="n">soup</span> <span class="o">=</span> <span class="nx">BeautifulSoup</span><span class="p">(</span><span class="s2">"<b></b>"</span><span class="p">)</span>
<span class="n">original_tag</span> <span class="o">=</span> <span class="nx">soup.b</span>
<span class="n">new_tag</span> <span class="o">=</span> <span class="nx">soup.new_tag</span><span class="p">(</span><span class="s2">"a"</span><span class="p">,</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://www.example.com"</span><span class="p">)</span>
<span class="nx">original_tag.append</span><span class="p">(</span><span class="nx">new_tag</span><span class="p">)</span>
<span class="nx">original_tag</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">b</span><span class="o">><</span><span class="nx">a</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://www.example.com"</span><span class="o">></</span><span class="nx">a</span><span class="o">></</span><span class="nx">b</span><span class="o">></span>
<span class="n">new_tag.string</span> <span class="o">=</span> <span class="s2">"Link text."</span>
<span class="nx">original_tag</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">b</span><span class="o">><</span><span class="nx">a</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://www.example.com"</span><span class="o">></span><span class="k">Link</span> <span class="nx">text.</span><span class="o"></</span><span class="nx">a</span><span class="o">></</span><span class="nx">b</span><span class="o">></span>
</pre></div>
<p>第一个参数作为tag的name,是必填,其它参数选填</p>
<h2 id="insert">insert()</h2>
<p><code>Tag.insert()</code> 方法与 <code>Tag.append()</code> 方法类似,区别是不会把新元素添加到父节点 <code>.contents</code> 属性的最后,而是把元素插入到指定的位置.与Python列表总的 <code>.insert()</code> 方法的用法下同:</p>
<div class="hlcode"><pre><span class="n">markup</span> <span class="o">=</span> <span class="s1">'<a href="http://example.com/">I linked to <i>example.com</i></a>'</span>
<span class="n">soup</span> <span class="o">=</span> <span class="nx">BeautifulSoup</span><span class="p">(</span><span class="nx">markup</span><span class="p">)</span>
<span class="kt">tag</span> <span class="o">=</span> <span class="nx">soup.a</span>
<span class="kt">tag</span><span class="bp">.</span><span class="nb">insert</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="s2">"but did not endorse "</span><span class="p">)</span>
<span class="kt">tag</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/"</span><span class="o">></span><span class="nx">I</span> <span class="nx">linked</span> <span class="k">to</span> <span class="nx">but</span> <span class="nx">did</span> <span class="ow">not</span> <span class="nx">endorse</span> <span class="o"><</span><span class="nx">i</span><span class="o">></span><span class="nx">example.com</span><span class="o"></</span><span class="nx">i</span><span class="o">></</span><span class="nx">a</span><span class="o">></span>
<span class="kt">tag</span><span class="bp">.</span><span class="nb">contents</span>
<span class="err">#</span> <span class="err">[</span><span class="nx">u</span><span class="s1">'I linked to '</span><span class="p">,</span> <span class="nx">u</span><span class="s1">'but did not endorse'</span><span class="p">,</span> <span class="o"><</span><span class="nx">i</span><span class="o">></span><span class="nx">example.com</span><span class="o"></</span><span class="nx">i</span><span class="o">></span><span class="cp">]</span>
</pre></div>
<h2 id="insert_before-insert_after">insert_before() 和 insert_after()</h2>
<p><code>insert_before()</code> 方法在当前tag或文本节点前插入内容:</p>
<div class="hlcode"><pre><span class="n">soup</span> <span class="o">=</span> <span class="nx">BeautifulSoup</span><span class="p">(</span><span class="s2">"<b>stop</b>"</span><span class="p">)</span>
<span class="kt">tag</span> <span class="o">=</span> <span class="nx">soup.new_tag</span><span class="p">(</span><span class="s2">"i"</span><span class="p">)</span>
<span class="kt">tag</span><span class="bp">.</span><span class="nb">string</span> <span class="o">=</span> <span class="s2">"Don't"</span>
<span class="nx">soup.b.string.insert_before</span><span class="p">(</span><span class="kt">tag</span><span class="p">)</span>
<span class="nx">soup.b</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">b</span><span class="o">><</span><span class="nx">i</span><span class="o">></span><span class="nx">Don</span><span class="s1">'t</i>stop</b></span>
</pre></div>
<p><code>insert_after()</code> 方法在当前tag或文本节点后插入内容:</p>
<div class="hlcode"><pre><span class="nx">soup.b.i.insert_after</span><span class="p">(</span><span class="nx">soup.new_string</span><span class="p">(</span><span class="s2">" ever "</span><span class="p">))</span>
<span class="nx">soup.b</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">b</span><span class="o">><</span><span class="nx">i</span><span class="o">></span><span class="nx">Don</span><span class="s1">'t</i> ever stop</b></span>
<span class="s1">soup.b.contents</span>
<span class="s1"># [<i>Don'</span><span class="nb">t</span><span class="o"></</span><span class="nx">i</span><span class="o">></span><span class="p">,</span> <span class="nx">u</span><span class="s1">' ever '</span><span class="p">,</span> <span class="nx">u</span><span class="s1">'stop'</span><span class="cp">]</span>
</pre></div>
<h2 id="clear">clear()</h2>
<p><code>Tag.clear()</code> 方法移除当前tag的内容:</p>
<div class="hlcode"><pre><span class="n">markup</span> <span class="o">=</span> <span class="s1">'<a href="http://example.com/">I linked to <i>example.com</i></a>'</span>
<span class="n">soup</span> <span class="o">=</span> <span class="nx">BeautifulSoup</span><span class="p">(</span><span class="nx">markup</span><span class="p">)</span>
<span class="kt">tag</span> <span class="o">=</span> <span class="nx">soup.a</span>
<span class="kt">tag</span><span class="bp">.</span><span class="nb">clear</span><span class="p">()</span>
<span class="kt">tag</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/"</span><span class="o">></</span><span class="nx">a</span><span class="o">></span>
</pre></div>
<h2 id="extract">extract()</h2>
<p><code>PageElement.extract()</code> 方法将当前tag移除文档树,并作为方法结果返回:</p>
<div class="hlcode"><pre><span class="n">markup</span> <span class="o">=</span> <span class="s1">'<a href="http://example.com/">I linked to <i>example.com</i></a>'</span>
<span class="n">soup</span> <span class="o">=</span> <span class="nx">BeautifulSoup</span><span class="p">(</span><span class="nx">markup</span><span class="p">)</span>
<span class="n">a_tag</span> <span class="o">=</span> <span class="nx">soup.a</span>
<span class="n">i_tag</span> <span class="o">=</span> <span class="nx">soup.i.extract</span><span class="p">()</span>
<span class="nx">a_tag</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/"</span><span class="o">></span><span class="nx">I</span> <span class="nx">linked</span> <span class="k">to</span><span class="o"></</span><span class="nx">a</span><span class="o">></span>
<span class="nx">i_tag</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">i</span><span class="o">></span><span class="nx">example.com</span><span class="o"></</span><span class="nx">i</span><span class="o">></span>
<span class="nx">print</span><span class="p">(</span><span class="nx">i_tag.parent</span><span class="p">)</span>
<span class="kc">None</span>
</pre></div>
<p>这个方法实际上产生了2个文档树: 一个是用来解析原始文档的 <code>BeautifulSoup</code> 对象,另一个是被移除并且返回的tag.被移除并返回的tag可以继续调用 <code>extract</code> 方法:</p>
<div class="hlcode"><pre><span class="n">my_string</span> <span class="o">=</span> <span class="nx">i_tag.string.extract</span><span class="p">()</span>
<span class="nx">my_string</span>
<span class="err">#</span> <span class="nx">u</span><span class="s1">'example.com'</span>
<span class="nx">print</span><span class="p">(</span><span class="nx">my_string.parent</span><span class="p">)</span>
<span class="err">#</span> <span class="kc">None</span>
<span class="nx">i_tag</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">i</span><span class="o">></</span><span class="nx">i</span><span class="o">></span>
</pre></div>
<h2 id="decompose">decompose()</h2>
<p><code>Tag.decompose()</code> 方法将当前节点移除文档树并完全销毁:</p>
<div class="hlcode"><pre><span class="n">markup</span> <span class="o">=</span> <span class="s1">'<a href="http://example.com/">I linked to <i>example.com</i></a>'</span>
<span class="n">soup</span> <span class="o">=</span> <span class="nx">BeautifulSoup</span><span class="p">(</span><span class="nx">markup</span><span class="p">)</span>
<span class="n">a_tag</span> <span class="o">=</span> <span class="nx">soup.a</span>
<span class="nx">soup.i.decompose</span><span class="p">()</span>
<span class="nx">a_tag</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/"</span><span class="o">></span><span class="nx">I</span> <span class="nx">linked</span> <span class="k">to</span><span class="o"></</span><span class="nx">a</span><span class="o">></span>
</pre></div>
<h2 id="replace_with">replace_with()</h2>
<p><code>PageElement.replace_with()</code> 方法移除文档树中的某段内容,并用新tag或文本节点替代它:</p>
<div class="hlcode"><pre><span class="n">markup</span> <span class="o">=</span> <span class="s1">'<a href="http://example.com/">I linked to <i>example.com</i></a>'</span>
<span class="n">soup</span> <span class="o">=</span> <span class="nx">BeautifulSoup</span><span class="p">(</span><span class="nx">markup</span><span class="p">)</span>
<span class="n">a_tag</span> <span class="o">=</span> <span class="nx">soup.a</span>
<span class="n">new_tag</span> <span class="o">=</span> <span class="nx">soup.new_tag</span><span class="p">(</span><span class="s2">"b"</span><span class="p">)</span>
<span class="n">new_tag.string</span> <span class="o">=</span> <span class="s2">"example.net"</span>
<span class="nx">a_tag.i.replace_with</span><span class="p">(</span><span class="nx">new_tag</span><span class="p">)</span>
<span class="nx">a_tag</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/"</span><span class="o">></span><span class="nx">I</span> <span class="nx">linked</span> <span class="k">to</span> <span class="o"><</span><span class="nx">b</span><span class="o">></span><span class="nx">example.net</span><span class="o"></</span><span class="nx">b</span><span class="o">></</span><span class="nx">a</span><span class="o">></span>
</pre></div>
<p><code>replace_with()</code> 方法返回被替代的tag或文本节点,可以用来浏览或添加到文档树其它地方</p>
<h2 id="wrap">wrap()</h2>
<p><code>PageElement.wrap()</code> 方法可以对指定的tag元素进行包装 [<a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#id89">8]</a> ,并返回包装后的结果:</p>
<div class="hlcode"><pre><span class="n">soup</span> <span class="o">=</span> <span class="nx">BeautifulSoup</span><span class="p">(</span><span class="s2">"<p>I wish I was bold.</p>"</span><span class="p">)</span>
<span class="nx">soup.p.string.wrap</span><span class="p">(</span><span class="nx">soup.new_tag</span><span class="p">(</span><span class="s2">"b"</span><span class="p">))</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">b</span><span class="o">></span><span class="nx">I</span> <span class="nx">wish</span> <span class="nx">I</span> <span class="nx">was</span> <span class="nx">bold.</span><span class="o"></</span><span class="nx">b</span><span class="o">></span>
<span class="nx">soup.p.wrap</span><span class="p">(</span><span class="nx">soup.new_tag</span><span class="p">(</span><span class="s2">"div"</span><span class="p">))</span>
<span class="err">#</span> <span class="o"><</span><span class="nb">div</span><span class="o">><</span><span class="nx">p</span><span class="o">><</span><span class="nx">b</span><span class="o">></span><span class="nx">I</span> <span class="nx">wish</span> <span class="nx">I</span> <span class="nx">was</span> <span class="nx">bold.</span><span class="o"></</span><span class="nx">b</span><span class="o">></</span><span class="nx">p</span><span class="o">></</span><span class="nb">div</span><span class="o">></span>
</pre></div>
<p>该方法在 Beautiful Soup 4.0.5 中添加</p>
<h2 id="unwrap">unwrap()</h2>
<p><code>Tag.unwrap()</code> 方法与 <code>wrap()</code> 方法相反.将移除tag内的所有tag标签,该方法常被用来进行标记的解包:</p>
<div class="hlcode"><pre><span class="n">markup</span> <span class="o">=</span> <span class="s1">'<a href="http://example.com/">I linked to <i>example.com</i></a>'</span>
<span class="n">soup</span> <span class="o">=</span> <span class="nx">BeautifulSoup</span><span class="p">(</span><span class="nx">markup</span><span class="p">)</span>
<span class="n">a_tag</span> <span class="o">=</span> <span class="nx">soup.a</span>
<span class="nx">a_tag.i.unwrap</span><span class="p">()</span>
<span class="nx">a_tag</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/"</span><span class="o">></span><span class="nx">I</span> <span class="nx">linked</span> <span class="k">to</span> <span class="nx">example.com</span><span class="o"></</span><span class="nx">a</span><span class="o">></span>
</pre></div>
<p>与 <code>replace_with()</code> 方法相同, <code>unwrap()</code> 方法返回被移除的tag</p>
<h1 id="soup">Soup输出</h1>
<h2 id="prettify">prettify</h2>
<p><code>prettify()</code> 方法将Beautiful Soup的文档树格式化后以Unicode编码输出,每个XML/HTML标签都独占一行</p>
<div class="hlcode"><pre><span class="n">markup</span> <span class="o">=</span> <span class="s1">'<a href="http://example.com/">I linked to <i>example.com</i></a>'</span>
<span class="n">soup</span> <span class="o">=</span> <span class="nx">BeautifulSoup</span><span class="p">(</span><span class="nx">markup</span><span class="p">)</span>
<span class="nx">soup.prettify</span><span class="p">()</span>
<span class="err">#</span> <span class="s1">'<html></span><span class="se">\n</span><span class="s1"> <head></span><span class="se">\n</span><span class="s1"> </head></span><span class="se">\n</span><span class="s1"> <body></span><span class="se">\n</span><span class="s1"> <a href="http://example.com/"></span><span class="se">\n</span><span class="s1">...'</span>
<span class="nx">print</span><span class="p">(</span><span class="nx">soup.prettify</span><span class="p">())</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">html</span><span class="o">></span>
<span class="err">#</span> <span class="o"><</span><span class="nb">head</span><span class="o">></span>
<span class="err">#</span> <span class="o"></</span><span class="nb">head</span><span class="o">></span>
<span class="err">#</span> <span class="o"><</span><span class="nb">body</span><span class="o">></span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/"</span><span class="o">></span>
<span class="err">#</span> <span class="nx">I</span> <span class="nx">linked</span> <span class="k">to</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">i</span><span class="o">></span>
<span class="err">#</span> <span class="nx">example.com</span>
<span class="err">#</span> <span class="o"></</span><span class="nx">i</span><span class="o">></span>
<span class="err">#</span> <span class="o"></</span><span class="nx">a</span><span class="o">></span>
<span class="err">#</span> <span class="o"></</span><span class="nb">body</span><span class="o">></span>
<span class="err">#</span> <span class="o"></</span><span class="nx">html</span><span class="o">></span>
</pre></div>
<p><code>BeautifulSoup</code> 对象和它的tag节点都可以调用 <code>prettify()</code> 方法:</p>
<div class="hlcode"><pre><span class="nx">print</span><span class="p">(</span><span class="nx">soup.a.prettify</span><span class="p">())</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/"</span><span class="o">></span>
<span class="err">#</span> <span class="nx">I</span> <span class="nx">linked</span> <span class="k">to</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">i</span><span class="o">></span>
<span class="err">#</span> <span class="nx">example.com</span>
<span class="err">#</span> <span class="o"></</span><span class="nx">i</span><span class="o">></span>
<span class="err">#</span> <span class="o"></</span><span class="nx">a</span><span class="o">></span>
</pre></div>
<h2 id="str-unicode">Str & Unicode输出</h2>
<p>如果只想得到结果字符串,不重视格式,那么可以对一个 <code>BeautifulSoup</code> 对象或 <code>Tag</code> 对象使用Python的 <code>unicode()</code> 或 <code>str()</code> 方法:</p>
<div class="hlcode"><pre><span class="nx">str</span><span class="p">(</span><span class="nx">soup</span><span class="p">)</span>
<span class="err">#</span> <span class="s1">'<html><head></head><body><a href="http://example.com/">I linked to <i>example.com</i></a></body></html>'</span>
<span class="nx">unicode</span><span class="p">(</span><span class="nx">soup.a</span><span class="p">)</span>
<span class="err">#</span> <span class="nx">u</span><span class="s1">'<a href="http://example.com/">I linked to <i>example.com</i></a>'</span>
</pre></div>
<blockquote>
<p><code>str()</code> 方法返回UTF-8编码的字符串,可以指定 <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#id51">编码</a> 的设置.</p>
<p>还可以调用 <code>encode()</code> 方法获得字节码或调用 <code>decode()</code> 方法获得Unicode.</p>
</blockquote>
<h2 id="get_text">get_text</h2>
<p>获取某个标签下的子孙非标签字符串，不是以列表形式返回，以普通字符串返回 </p>
<p>如果只想得到tag中包含的文本内容,那么可以嗲用 <code>get_text()</code> 方法,这个方法获取到tag中包含的所有文版内容包括子孙tag中的内容,并将结果作为Unicode字符串返回:</p>
<div class="hlcode"><pre><span class="n">markup</span> <span class="o">=</span> <span class="s1">'<a href="http://example.com/"></span><span class="se">\n</span><span class="s1">I linked to <i>example.com</i></span><span class="se">\n</span><span class="s1"></a>'</span>
<span class="n">soup</span> <span class="o">=</span> <span class="nx">BeautifulSoup</span><span class="p">(</span><span class="nx">markup</span><span class="p">)</span>
<span class="nx">soup.get_text</span><span class="p">()</span>
<span class="nx">u</span><span class="s1">'</span><span class="se">\n</span><span class="s1">I linked to example.com</span><span class="se">\n</span><span class="s1">'</span>
<span class="nx">soup.i.get_text</span><span class="p">()</span>
<span class="nx">u</span><span class="s1">'example.com'</span>
</pre></div>
<p>可以通过参数指定tag的文本内容的分隔符:</p>
<div class="hlcode"><pre><span class="cp"># soup.get_text("|")</span>
<span class="n">u</span><span class="err">'\</span><span class="n">nI</span> <span class="n">linked</span> <span class="n">to</span> <span class="o">|</span><span class="n">example</span><span class="p">.</span><span class="n">com</span><span class="o">|</span><span class="err">\</span><span class="n">n</span><span class="err">'</span>
</pre></div>
<p>还可以去除获得文本内容的前后空白:</p>
<div class="hlcode"><pre><span class="cp"># soup.get_text("|", strip=True)</span>
<span class="n">u</span><span class="err">'</span><span class="n">I</span> <span class="n">linked</span> <span class="n">to</span><span class="o">|</span><span class="n">example</span><span class="p">.</span><span class="n">com</span><span class="err">'</span>
</pre></div>
<p>或者使用 <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#strings-stripped-strings">.stripped_strings</a> 生成器,获得文本列表后手动处理列表:</p>
<div class="hlcode"><pre><span class="k">[text for text in soup.stripped_strings]</span>
<span class="c1"># [u'I linked to', u'example.com']</span>
</pre></div>
<h2 id="_12">输出格式</h2>
<p>Beautiful Soup输出是会将HTML中的特殊字符转换成Unicode,比如“&lquot;”:</p>
<div class="hlcode"><pre><span class="n">soup</span> <span class="o">=</span> <span class="nx">BeautifulSoup</span><span class="p">(</span><span class="s2">"“Dammit!” he said."</span><span class="p">)</span>
<span class="nx">unicode</span><span class="p">(</span><span class="nx">soup</span><span class="p">)</span> <span class="err">#</span> <span class="nx">Python</span> <span class="mi">2</span> <span class="nb">version</span>
<span class="err">#</span> <span class="nx">u</span><span class="s1">'<html><head></head><body></span><span class="se">\u201c</span><span class="s1">Dammit!</span><span class="se">\u201d</span><span class="s1"> he said.</body></html>'</span>
</pre></div>
<p>如果将文档转换成字符串,Unicode编码会被编码成UTF-8.这样就无法正确显示HTML特殊字符了:</p>
<div class="hlcode"><pre><span class="nx">str</span><span class="p">(</span><span class="nx">soup</span><span class="p">)</span>
<span class="err">#</span> <span class="s1">'<html><head></head><body></span><span class="se">\xe2\x80\x9c</span><span class="s1">Dammit!</span><span class="se">\xe2\x80\x9d</span><span class="s1"> he said.</body></html>'</span>
</pre></div>
<h2 id="_13">编码</h2>
<p>使用Beautiful Soup解析后,文档都被转换成了Unicode:</p>
<div class="hlcode"><pre><span class="n">markup</span> <span class="o">=</span> <span class="s2">"<h1>Sacr</span><span class="se">\xc3\xa9</span><span class="s2"> bleu!</h1>"</span>
<span class="n">soup</span> <span class="o">=</span> <span class="nx">BeautifulSoup</span><span class="p">(</span><span class="nx">markup</span><span class="p">)</span>
<span class="nx">soup.h1</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">h1</span><span class="o">></span><span class="nx">Sacré</span> <span class="nx">bleu</span><span class="o">!</</span><span class="nx">h1</span><span class="o">></span>
<span class="nx">soup.h1.string</span>
<span class="err">#</span> <span class="nx">u</span><span class="s1">'Sacr</span><span class="se">\xe9</span><span class="s1"> bleu!'</span>
</pre></div>
<p><code>BeautifulSoup</code> 对象的 <code>.original_encoding</code> 属性记录了自动识别编码的结果:</p>
<div class="hlcode"><pre><span class="n">soup</span><span class="p">.</span><span class="n">original_encoding</span>
<span class="err">'</span><span class="n">utf</span><span class="o">-</span><span class="mi">8</span><span class="err">'</span>
</pre></div>
<p><a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#unicode-dammit">编码自动检测</a> 功能大部分时候都能猜对编码格式,但有时候也会出错.有时候即使猜测正确,也是在逐个字节的遍历整个文档后才猜对的,这样很慢.如果预先知道文档编码,可以设置编码参数来减少自动检查编码出错的概率并且提高文档解析速度.在创建 <code>BeautifulSoup</code> 对象的时候设置 <code>from_encoding</code> 参数.</p>
<p>通过传入 <code>from_encoding</code> 参数来指定编码方式:</p>
<div class="hlcode"><pre><span class="n">soup</span> <span class="o">=</span> <span class="nx">BeautifulSoup</span><span class="p">(</span><span class="nx">markup</span><span class="p">,</span> <span class="n">from_encoding</span><span class="o">=</span><span class="s2">"iso-8859-8"</span><span class="p">)</span>
<span class="nx">soup.h1</span>
<span class="o"><</span><span class="nx">h1</span><span class="o">></span><span class="nx">םולש</span><span class="o"></</span><span class="nx">h1</span><span class="o">></span>
<span class="nx">soup.original_encoding</span>
<span class="s1">'iso8859-8'</span>
</pre></div>
<h3 id="_14">编码输出</h3>
<p>通过Beautiful Soup输出文档时,不管输入文档是什么编码方式,输出编码均为UTF-8编码,</p>
<p>如果不想用UTF-8编码输出,可以将编码方式传入 <code>prettify()</code> 方法:</p>
<div class="hlcode"><pre><span class="nx">print</span><span class="p">(</span><span class="nx">soup.prettify</span><span class="p">(</span><span class="s2">"latin-1"</span><span class="p">))</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">html</span><span class="o">></span>
<span class="err">#</span> <span class="o"><</span><span class="nb">head</span><span class="o">></span>
<span class="err">#</span> <span class="o"><</span><span class="nb">meta</span> <span class="n">content</span><span class="o">=</span><span class="s2">"text/html; charset=latin-1"</span> <span class="nx">http</span><span class="na">-equiv</span><span class="o">=</span><span class="s2">"Content-type"</span> <span class="o">/></span>
<span class="err">#</span> <span class="nx">...</span>
</pre></div>
<p>还可以调用 <code>BeautifulSoup</code> 对象或任意节点的 <code>encode()</code> 方法,就像Python的字符串调用 <code>encode()</code> 方法一样:</p>
<div class="hlcode"><pre><span class="nx">soup.p.encode</span><span class="p">(</span><span class="s2">"latin-1"</span><span class="p">)</span>
<span class="err">#</span> <span class="s1">'<p>Sacr</span><span class="se">\xe9</span><span class="s1"> bleu!</p>'</span>
<span class="nx">soup.p.encode</span><span class="p">(</span><span class="s2">"utf-8"</span><span class="p">)</span>
<span class="err">#</span> <span class="s1">'<p>Sacr</span><span class="se">\xc3\xa9</span><span class="s1"> bleu!</p>'</span>
</pre></div>
<h3 id="_15">检测编码</h3>
<p><a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#unicode-dammit">编码自动检测</a> 功能可以在Beautiful Soup以外使用,检测某段未知编码时,可以使用这个方法:</p>
<div class="hlcode"><pre><span class="n">from</span> <span class="n">bs4</span> <span class="n">import</span> <span class="n">UnicodeDammit</span>
<span class="n">dammit</span> <span class="o">=</span> <span class="n">UnicodeDammit</span><span class="p">(</span><span class="s">"Sacr</span><span class="se">\xc3\xa9</span><span class="s"> bleu!"</span><span class="p">)</span>
<span class="n">print</span><span class="p">(</span><span class="n">dammit</span><span class="p">.</span><span class="n">unicode_markup</span><span class="p">)</span>
<span class="cp"># Sacré bleu!</span>
<span class="n">dammit</span><span class="p">.</span><span class="n">original_encoding</span>
<span class="cp"># 'utf-8'</span>
</pre></div>
<p>如果Python中安装了 <code>chardet</code> 或 <code>cchardet</code> 那么编码检测功能的准确率将大大提高.输入的字符越多,检测结果越精确,如果事先猜测到一些可能编码,那么可以将猜测的编码作为参数,这样将优先检测这些编码:</p>
<div class="hlcode"><pre><span class="n">dammit</span> <span class="o">=</span> <span class="n">UnicodeDammit</span><span class="p">(</span><span class="s">"Sacr</span><span class="se">\xe9</span><span class="s"> bleu!"</span><span class="p">,</span> <span class="p">[</span><span class="s">"latin-1"</span><span class="p">,</span> <span class="s">"iso-8859-1"</span><span class="p">])</span>
<span class="n">print</span><span class="p">(</span><span class="n">dammit</span><span class="p">.</span><span class="n">unicode_markup</span><span class="p">)</span>
<span class="cp"># Sacré bleu!</span>
<span class="n">dammit</span><span class="p">.</span><span class="n">original_encoding</span>
<span class="cp"># 'latin-1'</span>
</pre></div>
<h1 id="_16">解析部分文档</h1>
<p>如果仅仅因为想要查找文档中的<a>标签而将整片文档进行解析,实在是浪费内存和时间.最快的方法是从一开始就把<a>标签以外的东西都忽略掉. <code>SoupStrainer</code> 类可以定义文档的某段内容,这样搜索文档时就不必先解析整篇文档,只会解析在 <code>SoupStrainer</code> 中定义过的文档. 创建一个 <code>SoupStrainer</code> 对象并作为 <code>parse_only</code> 参数给 <code>BeautifulSoup</code> 的构造方法即可.</p>
<h2 id="soupstrainer">SoupStrainer</h2>
<p><code>SoupStrainer</code> 类接受与典型搜索方法相同的参数：<a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#id32">name</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#css">attrs</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#recursive">recursive</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#text">text</a> , <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#keyword">**kwargs</a> 。下面举例说明三种 <code>SoupStrainer</code> 对象：</p>
<div class="hlcode"><pre><span class="n">from</span> <span class="n">bs4</span> <span class="n">import</span> <span class="n">SoupStrainer</span>
<span class="n">only_a_tags</span> <span class="o">=</span> <span class="n">SoupStrainer</span><span class="p">(</span><span class="s">"a"</span><span class="p">)</span>
<span class="n">only_tags_with_id_link2</span> <span class="o">=</span> <span class="n">SoupStrainer</span><span class="p">(</span><span class="n">id</span><span class="o">=</span><span class="s">"link2"</span><span class="p">)</span>
<span class="n">def</span> <span class="n">is_short_string</span><span class="p">(</span><span class="n">string</span><span class="p">)</span><span class="o">:</span>
<span class="k">return</span> <span class="n">len</span><span class="p">(</span><span class="n">string</span><span class="p">)</span> <span class="o"><</span> <span class="mi">10</span>
<span class="n">only_short_strings</span> <span class="o">=</span> <span class="n">SoupStrainer</span><span class="p">(</span><span class="n">text</span><span class="o">=</span><span class="n">is_short_string</span><span class="p">)</span>
</pre></div>
<p>再拿“爱丽丝”文档来举例，来看看使用三种 <code>SoupStrainer</code> 对象做参数会有什么不同:</p>
<div class="hlcode"><pre><span class="n">html_doc</span> <span class="o">=</span> <span class="s2">"""</span>
<span class="s2"><html><head><title>The Dormouse's story</title></head></span>
<span class="s2"><p class="</span><span class="nb">title</span><span class="s2">"><b>The Dormouse's story</b></p></span>
<span class="s2"><p class="</span><span class="nx">story</span><span class="s2">">Once upon a time there were three little sisters; and their names were</span>
<span class="s2"><a href="</span><span class="nx">http</span><span class="p">:</span><span class="c1">//example.com/elsie" class="sister" id="link1">Elsie</a>,</span>
<span class="o"><</span><span class="nx">a</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/lacie"</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link2"</span><span class="o">></span><span class="nx">Lacie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span> <span class="ow">and</span>
<span class="o"><</span><span class="nx">a</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/tillie"</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link3"</span><span class="o">></span><span class="nx">Tillie</span><span class="o"></</span><span class="nx">a</span><span class="o">></span><span class="p">;</span>
<span class="ow">and</span> <span class="nx">they</span> <span class="nx">lived</span> <span class="nx">at</span> <span class="nx">the</span> <span class="nx">bottom</span> <span class="nx">of</span> <span class="nx">a</span> <span class="nx">well.</span><span class="o"></</span><span class="nx">p</span><span class="o">></span>
<span class="o"><</span><span class="nx">p</span> <span class="n">class</span><span class="o">=</span><span class="s2">"story"</span><span class="o">></span><span class="nx">...</span><span class="o"></</span><span class="nx">p</span><span class="o">></span>
<span class="s2">"""</span>
<span class="s2">print(BeautifulSoup(html_doc, "</span><span class="nx">html.parser</span><span class="s2">", parse_only=only_a_tags).prettify())</span>
<span class="s2"># <a class="</span><span class="nx">sister</span><span class="s2">" href="</span><span class="nx">http</span><span class="p">:</span><span class="c1">//example.com/elsie" id="link1"></span>
<span class="err">#</span> <span class="nx">Elsie</span>
<span class="err">#</span> <span class="o"></</span><span class="nx">a</span><span class="o">></span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/lacie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link2"</span><span class="o">></span>
<span class="err">#</span> <span class="nx">Lacie</span>
<span class="err">#</span> <span class="o"></</span><span class="nx">a</span><span class="o">></span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/tillie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link3"</span><span class="o">></span>
<span class="err">#</span> <span class="nx">Tillie</span>
<span class="err">#</span> <span class="o"></</span><span class="nx">a</span><span class="o">></span>
<span class="nx">print</span><span class="p">(</span><span class="nx">BeautifulSoup</span><span class="p">(</span><span class="nx">html_doc</span><span class="p">,</span> <span class="s2">"html.parser"</span><span class="p">,</span> <span class="n">parse_only</span><span class="o">=</span><span class="nx">only_tags_with_id_link2</span><span class="p">)</span><span class="bp">.</span><span class="nx">prettify</span><span class="p">())</span>
<span class="err">#</span> <span class="o"><</span><span class="nx">a</span> <span class="n">class</span><span class="o">=</span><span class="s2">"sister"</span> <span class="n">href</span><span class="o">=</span><span class="s2">"http://example.com/lacie"</span> <span class="n">id</span><span class="o">=</span><span class="s2">"link2"</span><span class="o">></span>
<span class="err">#</span> <span class="nx">Lacie</span>
<span class="err">#</span> <span class="o"></</span><span class="nx">a</span><span class="o">></span>
<span class="nx">print</span><span class="p">(</span><span class="nx">BeautifulSoup</span><span class="p">(</span><span class="nx">html_doc</span><span class="p">,</span> <span class="s2">"html.parser"</span><span class="p">,</span> <span class="n">parse_only</span><span class="o">=</span><span class="nx">only_short_strings</span><span class="p">)</span><span class="bp">.</span><span class="nx">prettify</span><span class="p">())</span>
<span class="err">#</span> <span class="nx">Elsie</span>
<span class="err">#</span> <span class="p">,</span>
<span class="err">#</span> <span class="nx">Lacie</span>
<span class="err">#</span> <span class="ow">and</span>
<span class="err">#</span> <span class="nx">Tillie</span>
<span class="err">#</span> <span class="nx">...</span>
<span class="err">#</span>
</pre></div>
<p>还可以将 <code>SoupStrainer</code> 作为参数传入 <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#id24">搜索文档树</a> 中提到的方法.这可能不是个常用用法,所以还是提一下:</p>
<div class="hlcode"><pre><span class="n">soup</span> <span class="o">=</span> <span class="n">BeautifulSoup</span><span class="p">(</span><span class="n">html_doc</span><span class="p">)</span>
<span class="n">soup</span><span class="p">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">only_short_strings</span><span class="p">)</span>
<span class="cp"># [u'\n\n', u'\n\n', u'Elsie', u',\n', u'Lacie', u' and\n', u'Tillie',</span>
<span class="cp"># u'\n\n', u'...', u'\n']</span>
</pre></div>
<h1 id="example">Example</h1>
<h2 id="weathercomcn">weather.com.cn</h2>
<div class="hlcode"><pre><span class="nt">import</span> <span class="nt">requests</span>
<span class="nt">from</span> <span class="nt">bs4</span> <span class="nt">import</span> <span class="nt">BeautifulSoup</span>
<span class="nt">def</span> <span class="nt">parse_page</span><span class="o">(</span><span class="nt">url</span><span class="o">):</span>
<span class="nt">headers</span> <span class="o">=</span> <span class="p">{</span>
<span class="s2">"User-Agent"</span><span class="o">:</span> <span class="s2">"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Iridium/2017.11 Safari/537.36 Chrome/62.0.3202.94"</span>
<span class="p">}</span>
<span class="nt">response</span> <span class="o">=</span> <span class="nt">requests</span><span class="nc">.get</span><span class="o">(</span><span class="nt">url</span><span class="o">,</span> <span class="nt">headers</span><span class="o">=</span><span class="nt">headers</span><span class="o">)</span>
<span class="nt">text</span> <span class="o">=</span> <span class="nt">response</span><span class="nc">.content.decode</span><span class="o">(</span><span class="s1">'utf-8'</span><span class="o">)</span>
<span class="nt">soup</span> <span class="o">=</span> <span class="nt">BeautifulSoup</span><span class="o">(</span><span class="nt">text</span><span class="o">,</span> <span class="s1">'html5lib'</span><span class="o">)</span>
<span class="nt">conMidtab</span> <span class="o">=</span> <span class="nt">soup</span><span class="nc">.find</span><span class="o">(</span><span class="s1">'div'</span><span class="o">,</span> <span class="nt">class_</span><span class="o">=</span><span class="s2">"conMidtab"</span><span class="o">)</span>
<span class="nt">tables</span> <span class="o">=</span> <span class="nt">conMidtab</span><span class="nc">.find_all</span><span class="o">(</span><span class="s1">'table'</span><span class="o">)</span>
<span class="nt">for</span> <span class="nt">table</span> <span class="nt">in</span> <span class="nt">tables</span><span class="o">:</span>
<span class="nt">trs</span> <span class="o">=</span> <span class="nt">table</span><span class="nc">.find_all</span><span class="o">(</span><span class="s1">'tr'</span><span class="o">)</span><span class="cp">[</span><span class="mi">2</span><span class="p">:</span><span class="cp">]</span>
<span class="nt">for</span> <span class="nt">index</span><span class="o">,</span> <span class="nt">tr</span> <span class="nt">in</span> <span class="nt">enumerate</span><span class="o">(</span><span class="nt">trs</span><span class="o">):</span>
<span class="nt">tds</span> <span class="o">=</span> <span class="nt">tr</span><span class="nc">.find_all</span><span class="o">(</span><span class="s1">'td'</span><span class="o">)</span>
<span class="nt">city_td</span> <span class="o">=</span> <span class="nt">tds</span><span class="cp">[</span><span class="mi">0</span><span class="cp">]</span>
<span class="nt">if</span> <span class="nt">index</span> <span class="o">==</span> <span class="nt">0</span><span class="o">:</span>
<span class="nt">city_td</span> <span class="o">=</span> <span class="nt">tds</span><span class="cp">[</span><span class="mi">1</span><span class="cp">]</span>
<span class="nt">city</span> <span class="o">=</span> <span class="nt">list</span><span class="o">(</span><span class="nt">city_td</span><span class="nc">.stripped_strings</span><span class="o">)</span><span class="cp">[</span><span class="mi">0</span><span class="cp">]</span>
<span class="nt">high_temp_td</span> <span class="o">=</span> <span class="nt">tds</span><span class="cp">[</span><span class="o">-</span><span class="mi">5</span><span class="cp">]</span>
<span class="nt">high_temp</span> <span class="o">=</span> <span class="nt">list</span><span class="o">(</span><span class="nt">high_temp_td</span><span class="nc">.stripped_strings</span><span class="o">)</span><span class="cp">[</span><span class="mi">0</span><span class="cp">]</span>
<span class="nt">low_temp_td</span> <span class="o">=</span> <span class="nt">tds</span><span class="cp">[</span><span class="o">-</span><span class="mi">2</span><span class="cp">]</span>
<span class="nt">low_temp</span> <span class="o">=</span> <span class="nt">list</span><span class="o">(</span><span class="nt">low_temp_td</span><span class="nc">.stripped_strings</span><span class="o">)</span><span class="cp">[</span><span class="mi">0</span><span class="cp">]</span>
<span class="nt">print</span><span class="o">(</span><span class="nt">city</span><span class="o">,</span> <span class="nt">high_temp</span><span class="o">,</span> <span class="nt">low_temp</span><span class="o">)</span>
<span class="nt">def</span> <span class="nt">main</span><span class="o">():</span>
<span class="nt">urls</span> <span class="o">=</span> <span class="cp">[</span>
<span class="s1">'http://www.weather.com.cn/textFC/hb.shtml'</span><span class="p">,</span>
<span class="s1">'http://www.weather.com.cn/textFC/db.shtml'</span><span class="p">,</span>
<span class="s1">'http://www.weather.com.cn/textFC/hd.shtml'</span><span class="p">,</span>
<span class="s1">'http://www.weather.com.cn/textFC/hz.shtml'</span><span class="p">,</span>
<span class="s1">'http://www.weather.com.cn/textFC/hn.shtml'</span><span class="p">,</span>
<span class="s1">'http://www.weather.com.cn/textFC/xb.shtml'</span><span class="p">,</span>
<span class="s1">'http://www.weather.com.cn/textFC/xn.shtml'</span><span class="p">,</span>
<span class="s1">'http://www.weather.com.cn/textFC/gat.shtml'</span>
<span class="cp">]</span>
<span class="nt">for</span> <span class="nt">url</span> <span class="nt">in</span> <span class="nt">urls</span><span class="o">:</span>
<span class="nt">parse_page</span><span class="o">(</span><span class="nt">url</span><span class="o">)</span>
<span class="nt">if</span> <span class="nt">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="o">:</span>
<span class="nt">main</span><span class="o">()</span>
</pre></div>
</div>
<div id="footer">
<span>
<p>Copyright © 2020 Xu XueHua.
Powered by <a href="http://simiki.org/" target="_blank">Simiki</a>.</p>
<p>Site Generated 2020-01-24 16:55:26</p>
</span>
</div>
</body>
</html>