前言
最近抽空想学习一下python的爬虫框架scrapy,在mac下安装的时候遇到了问题,逐一解决了问题,分享一下,话不多说了,来一起看看详细的介绍吧。
步骤如下:
1. 从官网 下载最新版本Python 3.6.3(本地快速下载安装:https://www.jb51.net/softs/583651.html)
# 在Mac上Python3环境下安装scrapy
2. 安装 Python3
在终端输入python3出现下面的内容表示安装成功
"help", "copyright", "credits" or "license" for more information. >
输入quit()
退出编辑模式
3. 输入 pip install scrapy
执行 scrapy 安装
"/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/basecommand.py", line 215, in main status = self.run(options, args) File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/commands/install.py", line 342, in run prefix=options.prefix_path, File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_set.py", line 784, in install **kwargs File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_install.py", line 851, in install self.move_wheel_files(self.source_dir, root=root, prefix=prefix) File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_install.py", line 1064, in move_wheel_files isolated=self.isolated, File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/wheel.py", line 345, in move_wheel_files clobber(source, lib_dir, True) File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/wheel.py", line 316, in clobber ensure_dir(destdir) File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/utils/__init__.py", line 83, in ensure_dir os.makedirs(path) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 157, in makedirs mkdir(name, mode) OSError: [Errno 13] Permission denied: '/Library/Python/2.7/site-packages/lxml'
出现 OSError: [Errno 13] Permission denied: '/Library/Python/2.7/site-packages/lxml'
错误
4. 尝试重新安装lxml,执行 sudo pip install lxml
"htmlcode">"htmlcode">"/usr/local/bin/scrapy", line 7, in <module> from scrapy.cmdline import execute File "/Library/Python/2.7/site-packages/scrapy/cmdline.py", line 9, in <module> from scrapy.crawler import CrawlerProcess File "/Library/Python/2.7/site-packages/scrapy/crawler.py", line 7, in <module> from twisted.internet import reactor, defer File "/Library/Python/2.7/site-packages/twisted/internet/reactor.py", line 38, in <module> from twisted.internet import default File "/Library/Python/2.7/site-packages/twisted/internet/default.py", line 56, in <module> install = _getInstallFunction(platform) File "/Library/Python/2.7/site-packages/twisted/internet/default.py", line 50, in _getInstallFunction from twisted.internet.selectreactor import install File "/Library/Python/2.7/site-packages/twisted/internet/selectreactor.py", line 18, in <module> from twisted.internet import posixbase File "/Library/Python/2.7/site-packages/twisted/internet/posixbase.py", line 18, in <module> from twisted.internet import error, udp, tcp File "/Library/Python/2.7/site-packages/twisted/internet/tcp.py", line 28, in <module> from twisted.internet._newtls import ( File "/Library/Python/2.7/site-packages/twisted/internet/_newtls.py", line 21, in <module> from twisted.protocols.tls import TLSMemoryBIOFactory, TLSMemoryBIOProtocol File "/Library/Python/2.7/site-packages/twisted/protocols/tls.py", line 63, in <module> from twisted.internet._sslverify import _setAcceptableProtocols File "/Library/Python/2.7/site-packages/twisted/internet/_sslverify.py", line 38, in <module> TLSVersion.TLSv1_1: SSL.OP_NO_TLSv1_1, AttributeError: 'module' object has no attribute 'OP_NO_TLSv1_1'需要更新 OpenSSL 库,执行
sudo pip install --upgrade pyopenssl
"PyPy" (from cryptography>=1.9->pyopenssl) Downloading cffi-1.11.2-cp27-cp27m-macosx_10_6_intel.whl (238kB) 100% |████████████████████████████████| 245kB 2.2MB/s Collecting enum34; python_version < "3" (from cryptography>=1.9->pyopenssl) Downloading enum34-1.1.6-py2-none-any.whl Collecting idna>=2.1 (from cryptography>=1.9->pyopenssl) Downloading idna-2.6-py2.py3-none-any.whl (56kB) 100% |████████████████████████████████| 61kB 3.1MB/s Collecting asn1crypto>=0.21.0 (from cryptography>=1.9->pyopenssl) Downloading asn1crypto-0.23.0-py2.py3-none-any.whl (99kB) 100% |████████████████████████████████| 102kB 2.7MB/s Collecting ipaddress; python_version < "3" (from cryptography>=1.9->pyopenssl) Downloading ipaddress-1.0.18-py2-none-any.whl Collecting pycparser (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.9->pyopenssl) Downloading pycparser-2.18.tar.gz (245kB) 100% |████████████████████████████████| 256kB 3.6MB/s Installing collected packages: pycparser, cffi, enum34, idna, asn1crypto, ipaddress, cryptography, pyopenssl Running setup.py install for pycparser ... done Found existing installation: pyOpenSSL 0.13.1 DEPRECATION: Uninstalling a distutils installed project (pyopenssl) has been deprecated and will be removed in a future version. This is due to the fact that uninstalling a distutils project will only partially uninstall the project. Uninstalling pyOpenSSL-0.13.1: Successfully uninstalled pyOpenSSL-0.13.1 Successfully installed asn1crypto-0.23.0 cffi-1.11.2 cryptography-2.1.1 enum34-1.1.6 idna-2.6 ipaddress-1.0.18 pycparser-2.18 pyopenssl-17.3.0更新 OpenSSL 成功,再次尝试执行 scrapy
"scrapy <command> -h" to see more info about a command出现上面内容,表明安装成功。现在可以通过 scrapy 创建一个爬虫项目了
7. 进入到你项目的目录,执行
scrapy startproject firstscrapy
创建 firstscrapy 爬虫项目华山资源网 Design By www.eoogi.com
广告合作:本站广告合作请联系QQ:858582 申请时备注:广告合作(否则不回)
免责声明:本站资源来自互联网收集,仅供用于学习和交流,请遵循相关法律法规,本站一切资源不代表本站立场,如有侵权、后门、不妥请联系本站删除!
免责声明:本站资源来自互联网收集,仅供用于学习和交流,请遵循相关法律法规,本站一切资源不代表本站立场,如有侵权、后门、不妥请联系本站删除!
华山资源网 Design By www.eoogi.com
暂无评论...
更新日志
2024年11月18日
2024年11月18日
- 【雨果唱片】中国管弦乐《鹿回头》WAV
- APM亚流新世代《一起冒险》[FLAC/分轨][106.77MB]
- 崔健《飞狗》律冻文化[WAV+CUE][1.1G]
- 罗志祥《舞状元 (Explicit)》[320K/MP3][66.77MB]
- 尤雅.1997-幽雅精粹2CD【南方】【WAV+CUE】
- 张惠妹.2007-STAR(引进版)【EMI百代】【WAV+CUE】
- 群星.2008-LOVE情歌集VOL.8【正东】【WAV+CUE】
- 罗志祥《舞状元 (Explicit)》[FLAC/分轨][360.76MB]
- Tank《我不伟大,至少我能改变我。》[320K/MP3][160.41MB]
- Tank《我不伟大,至少我能改变我。》[FLAC/分轨][236.89MB]
- CD圣经推荐-夏韶声《谙2》SACD-ISO
- 钟镇涛-《百分百钟镇涛》首批限量版SACD-ISO
- 群星《继续微笑致敬许冠杰》[低速原抓WAV+CUE]
- 潘秀琼.2003-国语难忘金曲珍藏集【皇星全音】【WAV+CUE】
- 林东松.1997-2039玫瑰事件【宝丽金】【WAV+CUE】