
Python
wengyupeng
在等待中爆发
展开
-
'django-admin.py' is not recognized as an internal or external command
'django-admin.py' is not recognized as an internal or external command原创 2016-08-02 17:08:03 · 4496 阅读 · 0 评论 -
Python(pyspark) only supports DataFrames and not RDDs
一、背景用Mongo Spark Connector 来连接 python(pyspark)和MongoDB:二、问题报下面错误:Py4JJavaError: An error occurred while calling z:org.apache.spark.api.python.PythonRDD.saveAsNewAPIHadoopFile.: java.l...原创 2019-07-12 16:34:16 · 282 阅读 · 1 评论 -
通过mongo-hadoop(pymongo_spark)从PySpark保存数据到MongoDB
一、背景PySpark to connect to MongoDB via mongo-hadoop二、配置步骤 (注意版本作相应调整,spark-2.4.3,hadoop2.7,Scala2.11)1. # Get the MongoDB Java Driver#PROJECT_HOME 自定义的项目根目录,下面存放spark等mkdir -p $PR...原创 2019-07-12 17:58:16 · 1209 阅读 · 0 评论 -
spark:pyspark shell python tab自动提示
1. home dir 下创建.pythonrc ~> vi .pythonrcimport rlcompleter, readlinereadline.parse_and_bind('tab: complete')2. 在shell的启动文件添加.pythonrc 查看是那种shell。 如果是csh , startup file是.cshr...原创 2019-06-04 16:23:22 · 758 阅读 · 0 评论 -
Alpine pyspark ModuleNotFoundError: No module named 'zlib'
一、问题(在Alpine 上装好spark, 运行 pyspark)bash-4.4# pysparkPython 3.6.5 (default, May 30 2019, 09:48:14)[GCC 6.4.0] on linuxType "help", "copyright", "credits" or "license" for more information.Traceb...原创 2019-05-31 16:24:10 · 7210 阅读 · 0 评论 -
Alpine Linux 安装 python3.6
1: 下载 Code$ wget https://www.python.org/ftp/python/3.6.5/Python-3.6.5.tgz2: 准备环境//gcc --version 检查是否已经安装 ,若 bash: gcc: command not found 说明没安装apk add build-base //install GCC on...原创 2019-05-31 16:09:28 · 14094 阅读 · 3 评论 -
Python in worker has different version 2.7 than that in driver 3.6
一、问题Exception: Python in worker has different version 2.7 than that in driver 3.6, PySpark cannot run with different minor versions.Please check environment variables PYSPARK_PYTHON and PYSPARK_DRIV...原创 2019-06-05 10:21:12 · 2406 阅读 · 0 评论 -
shell 下python/PySpark 代码高亮和 自动提示 代码补全
1. 用法1. pip install "ptpython==0.41"#默认最新版本 ptpython-2.0.4 有bug, 自动提示不起作用。0.41的没问题#pip版本(pip --version),决定你装的库基于python2 还是3.2. export PYSPARK_DRIVER_PYTHON=ptpython;...原创 2019-06-04 16:56:23 · 1300 阅读 · 0 评论 -
Mac安装Django
imac 安装Django原创 2016-08-21 10:00:03 · 4275 阅读 · 0 评论 -
Mac 安装Scrapy
Mac 安装Scrapy原创 2016-09-03 10:02:52 · 1057 阅读 · 0 评论 -
403 CSRF verification failed. Request aborted
403 CSRF verification failed. Request aborted原创 2016-08-21 21:42:07 · 573 阅读 · 0 评论 -
Pycharm 创建 Django admin 用户名和密码
Pycharm 创建 Django admin 用户名和密码原创 2016-08-21 12:38:48 · 31936 阅读 · 1 评论 -
Jupyter Notebook 字体设置& 代码自动提示补全
作者:湫兮出处:https://www.cnblogs.com/qiuxirufeng/p/9609031.html1.首先是主题下载,命令行如下所示:pip install --no-dependencies jupyterthemes==0.18.22. 安装好了,有的电脑可能会提示缺少 lesscpy,继续 pip 安装pip install lesscpy...转载 2019-07-12 16:55:14 · 3278 阅读 · 0 评论