前言
最近抽空想学习一下python的爬虫框架scrapy,在mac下安装的时候遇到了问题,逐一解决了问题,分享一下,话不多说了,来一起看看详细的介绍吧。
步骤如下:
1. 从官网 下载最新版本Python 3.6.3(本地快速下载安装:https://www.jb51.net/softs/583651.html)
# 在Mac上Python3环境下安装scrapy
2. 安装 Python3
在终端输入python3出现下面的内容表示安装成功
"help", "copyright", "credits" or "license" for more information. >
输入quit()
退出编辑模式
3. 输入 pip install scrapy
执行 scrapy 安装
"/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/basecommand.py", line 215, in main status = self.run(options, args) File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/commands/install.py", line 342, in run prefix=options.prefix_path, File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_set.py", line 784, in install **kwargs File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_install.py", line 851, in install self.move_wheel_files(self.source_dir, root=root, prefix=prefix) File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_install.py", line 1064, in move_wheel_files isolated=self.isolated, File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/wheel.py", line 345, in move_wheel_files clobber(source, lib_dir, True) File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/wheel.py", line 316, in clobber ensure_dir(destdir) File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/utils/__init__.py", line 83, in ensure_dir os.makedirs(path) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 157, in makedirs mkdir(name, mode) OSError: [Errno 13] Permission denied: '/Library/Python/2.7/site-packages/lxml'
出现 OSError: [Errno 13] Permission denied: '/Library/Python/2.7/site-packages/lxml'
错误
4. 尝试重新安装lxml,执行 sudo pip install lxml
"htmlcode">"htmlcode">"/usr/local/bin/scrapy", line 7, in <module> from scrapy.cmdline import execute File "/Library/Python/2.7/site-packages/scrapy/cmdline.py", line 9, in <module> from scrapy.crawler import CrawlerProcess File "/Library/Python/2.7/site-packages/scrapy/crawler.py", line 7, in <module> from twisted.internet import reactor, defer File "/Library/Python/2.7/site-packages/twisted/internet/reactor.py", line 38, in <module> from twisted.internet import default File "/Library/Python/2.7/site-packages/twisted/internet/default.py", line 56, in <module> install = _getInstallFunction(platform) File "/Library/Python/2.7/site-packages/twisted/internet/default.py", line 50, in _getInstallFunction from twisted.internet.selectreactor import install File "/Library/Python/2.7/site-packages/twisted/internet/selectreactor.py", line 18, in <module> from twisted.internet import posixbase File "/Library/Python/2.7/site-packages/twisted/internet/posixbase.py", line 18, in <module> from twisted.internet import error, udp, tcp File "/Library/Python/2.7/site-packages/twisted/internet/tcp.py", line 28, in <module> from twisted.internet._newtls import ( File "/Library/Python/2.7/site-packages/twisted/internet/_newtls.py", line 21, in <module> from twisted.protocols.tls import TLSMemoryBIOFactory, TLSMemoryBIOProtocol File "/Library/Python/2.7/site-packages/twisted/protocols/tls.py", line 63, in <module> from twisted.internet._sslverify import _setAcceptableProtocols File "/Library/Python/2.7/site-packages/twisted/internet/_sslverify.py", line 38, in <module> TLSVersion.TLSv1_1: SSL.OP_NO_TLSv1_1, AttributeError: 'module' object has no attribute 'OP_NO_TLSv1_1'需要更新 OpenSSL 库,执行
sudo pip install --upgrade pyopenssl
"PyPy" (from cryptography>=1.9->pyopenssl) Downloading cffi-1.11.2-cp27-cp27m-macosx_10_6_intel.whl (238kB) 100% |████████████████████████████████| 245kB 2.2MB/s Collecting enum34; python_version < "3" (from cryptography>=1.9->pyopenssl) Downloading enum34-1.1.6-py2-none-any.whl Collecting idna>=2.1 (from cryptography>=1.9->pyopenssl) Downloading idna-2.6-py2.py3-none-any.whl (56kB) 100% |████████████████████████████████| 61kB 3.1MB/s Collecting asn1crypto>=0.21.0 (from cryptography>=1.9->pyopenssl) Downloading asn1crypto-0.23.0-py2.py3-none-any.whl (99kB) 100% |████████████████████████████████| 102kB 2.7MB/s Collecting ipaddress; python_version < "3" (from cryptography>=1.9->pyopenssl) Downloading ipaddress-1.0.18-py2-none-any.whl Collecting pycparser (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.9->pyopenssl) Downloading pycparser-2.18.tar.gz (245kB) 100% |████████████████████████████████| 256kB 3.6MB/s Installing collected packages: pycparser, cffi, enum34, idna, asn1crypto, ipaddress, cryptography, pyopenssl Running setup.py install for pycparser ... done Found existing installation: pyOpenSSL 0.13.1 DEPRECATION: Uninstalling a distutils installed project (pyopenssl) has been deprecated and will be removed in a future version. This is due to the fact that uninstalling a distutils project will only partially uninstall the project. Uninstalling pyOpenSSL-0.13.1: Successfully uninstalled pyOpenSSL-0.13.1 Successfully installed asn1crypto-0.23.0 cffi-1.11.2 cryptography-2.1.1 enum34-1.1.6 idna-2.6 ipaddress-1.0.18 pycparser-2.18 pyopenssl-17.3.0更新 OpenSSL 成功,再次尝试执行 scrapy
"scrapy <command> -h" to see more info about a command出现上面内容,表明安装成功。现在可以通过 scrapy 创建一个爬虫项目了
7. 进入到你项目的目录,执行
scrapy startproject firstscrapy
创建 firstscrapy 爬虫项目华山资源网 Design By www.eoogi.com
广告合作:本站广告合作请联系QQ:858582 申请时备注:广告合作(否则不回)
免责声明:本站资源来自互联网收集,仅供用于学习和交流,请遵循相关法律法规,本站一切资源不代表本站立场,如有侵权、后门、不妥请联系本站删除!
免责声明:本站资源来自互联网收集,仅供用于学习和交流,请遵循相关法律法规,本站一切资源不代表本站立场,如有侵权、后门、不妥请联系本站删除!
华山资源网 Design By www.eoogi.com
暂无评论...
更新日志
2024年09月25日
2024年09月25日
- 纯音入心系列纯音乐《天籁古筝》1CD[MP3][331MB]
- 男女对唱典藏天碟《发烧对唱·那个季节里的歌DSD》10CD[WAV]
- 群星2010-歌林精选辑[歌林][WAV+CUE]
- TrioZimbalist-PianoTriosofWeinberg,Auerbach,Dvorak(2024)[24-44,1]wav
- 群星.1992-滚石第一流台湾歌·12王牌大车拼【滚石】【WAV+CUE】
- 林子祥.1984-创作歌集【华纳】【WAV+CUE】
- 周汤豪.2010-周汤豪【华纳】【FLAC分轨】
- Mozart-TheStringQuintets-AmadeusQuartet,CecilAronowitz(2017)[24-44,1][WAV+CUE]
- JamesWilliamsDennisIrwin-Focus(2024,Red)[24-48]FLAC
- 藤泽麻衣《空みあげて》[WAV]
- 纯音入心系列纯音乐《中国古筝经典名曲》1CD[MP3][1.3GB]
- 纯音入心系列纯音乐《古筝新奏:岁月静好与筝语,细水流年与筝同》1CD[MP3][846.9MB]
- 670《脆弱敏感小女生》[320K/MP3][27.53MB]
- 曾琳.2011-好好爱我【南方】【WAV+CUE】
- 范晓萱.1995-RAIN【福茂】【WAV+CUE】