mzg12345678-优快云博客

翻译 DoubleArray词典管理

49 class PosWeightDict 50 { 51 public: 52 ~PosWeightDict(); 53 bool GetWeightInfo(std::string literal,WeightInfo* wei); 54 55 private: 56 uint8_t *m_map_ptr; 57 size_t m_map_s...

2018-08-29 18:18:37 219

翻译 Bsearch 词典管理

#include <assert.h>#include <sys/time.h>#include <openssl/md5.h>#include <algorithm>#include <stdint.h>#include <math.h>#include "include/share/darts.h"

2018-08-29 18:15:10 209

转载 Paper List

A Bag of Features for Short Text Classification Intent Based Relevance Estimation from Click Logs A Large Scale Prediction Engine for App Install Clicks and Conversions DeepRank: A New D...

2018-08-11 22:09:42 312

翻译 SynonymFilter

import tensorflow as tfimport numpy as npimport scipy.io as ioimport sysimport os# Define the flags useable from the command line.tf.app.flags.DEFINE_string('train', None, 'F...

2018-03-06 21:46:20 435

原创 POS_IDF

#! /usr/bin/pythonimport osimport sysfrom math import *Delta = 0.1npos_freq = {}npos_file = open("./npos_freq.dat")C = 0for line in npos_file: line = line.strip() if not line: continue ...

2018-03-06 15:27:03 184

翻译 pscp pssh Code

3 rm -rf ../collect_synom 4 mkdir -p ../collect_synom 5 6 handle_result () { 7 if [ $1 -ne 0 ]; then 8 echo "Job Failed; $2" 9 echo "Job Failed; $2" | mail -s "lsi follow f...

2018-03-05 18:27:42 371

原创特征重要性分析

1 from sklearn.tree import DecisionTreeRegressor 2 from sklearn.ensemble import RandomForestRegressor 3 import numpy as np 4 5 from sklearn.externals.joblib import Memory 6 from sklearn.datase...

2018-03-02 11:43:45 7813

翻译 ConvertProtoToArray

427 template 428 void MRFModel::ConvertProtoToArray(429 const google::protobuf::Message* message, std::vector* feature_array, std::string& ori) {430 feature_array->clear();431 ori = ""

2018-01-18 18:15:15 175

翻译 hadoop streaming 使用自己的 python 版本

1 #!/usr/bin/env python3 15 hadoop jar hadoop-streaming.jar \ 16 -Dmapred.fairscheduler.pool=build \ 17 -Dmapred.reduce.tasks=500 \ 18 -Dmapred.job.priority=VERY_HIGH \ 19 -Dmapred.job.nam

2018-01-05 19:13:47 2623

翻译 SVD MPI 实现

run_lsi.sh 1 ROOT_PATH=/da1/search/hpc_...

2018-01-03 16:23:42 327

翻译 TextRank Local

#! -*- coding:utf-8 -*-import numpy as np import jieba import jieba.posseg as pseg class TextRank(object): def __init__(self, sentence, window, alpha, iternum):

2017-12-28 11:39:23 256

翻译 Web_parser

# -*- coding:utf-8 _*-import sysimport osimport mathimport threadingimport urllibimport urllib2import reimport threadingimport socketimport codecsimport timefrom mu

2017-12-14 17:13:10 255

原创 TextRank Spark 实现

#! -*- coding:utf-8 -*-import osimport sysimport mathfrom pyspark import SparkContextfrom pyspark import StorageLevelfrom pyspark.conf import SparkConf #os.environ['PYSPARK_PYTHON']

2017-11-23 10:47:09 1128

翻译 darts_builder

#include <stdlib.h>#include <sys/mman.h>#include <stdio.h>#include <vector>#include <unistd.h>#include <stdint.h>using namespace std;#pragma pack(1)struct PhraseGro...

2017-11-08 11:21:00 255

翻译 Tensorflow Code 示例

1.张量乘法def f1(X,w): c = tf.einsum('ijl,lk->ijk', X, w) print c.shape return cdef output(self, x): batch_size = tf.shape(x)[0] x = tf.reshape(x, [-1, self._shape[0]])

2017-10-23 15:45:40 496

翻译 python Code 示例

1. numpy load data def extract_data(filename): out = np.loadtxt(filename, dtype=np.str,delimiter=" ",comments=None); # Arrays to hold the labels and feature vectors. labels = out[...

2017-10-21 17:57:08 656

翻译 scikit-learn SVM

import osimport pickle import sklearnfrom sklearn import cross_validation, grid_searchfrom sklearn.metrics import confusion_matrix, classification_reportfrom sklearn.svm import SVCfrom skl

2017-10-20 15:15:26 346

翻译 Scala 常用命令

1. def fabs(n: Double): Double = { if (n > 0) n else -n }2. def Cosin(vecA: Array[Float], vecB: Array[Float]): Double = { val product = vecA.view.zip(vecB.view).map

2017-10-18 16:47:17 1957

翻译 shell 常用命令

1. svn st | awk '{if ( $1 == "C") { print $2}}' | xargs svn resolve --accept working2. hadoop fs -ls /home/| grep "log" | tail -60 | awk -F"/" '{if(NF>=3){printf("%s ",\$NF)}}3. 0 8 * * * cd /

2017-10-18 11:57:19 179

翻译 python 多线程计算特征

#! /usr/bin/env python# -*- coding:utf-8 -*-import sysimport osfrom multiprocessing import Process, Lock, Queue, Managerfrom multiprocessing.managers import BaseManagerimport argpa

2017-10-17 11:57:36 473

翻译文本对齐

#!/usr/bin/env python# -*- coding: utf-8 -*-def reverse(ans): res = [] for i in range(len(ans),0,-1): res.append(ans[i-1]) return resdef

2017-10-13 12:10:41 355

翻译汉字处理的工具

#!/usr/bin/env python# -*- coding:utf-8 -*-"""汉字处理的工具:判断unicode是否是汉字，数字，英文，或者其他字符。全角符号转半角符号。"""def is_chinese(uchar): """判断一个unicode是否是汉字""" if uchar >= u'\u4e00' and uchar

2017-10-13 10:24:01 414