<> concept
According to your “ neighbor ” Judge your category
<> technological process
<>KNN api Preliminary use
<> Machine learning process
<>Scikit-learn tool
<> install
pip3 install scikit-learn==0.19.1
notes ： need Numpy,Scipy Wait for library support
Python (>= 3.5),
NumPy (>= 1.11.0),
SciPy (>= 0.17.0),
joblib (>= 0.11).
<> Inspection and installation
import sklearn
<>Scikit-learn Include content
<>K- Nearest neighbor algorithm API
<> test
from sklearn.neighbors import KNeighborsClassifier # Construction data x = [[1], [5], [10],
[20]] y = [2, 2, 6, 6] # Training model # Instantiate estimator object estimator = KNeighborsClassifier(
n_neighbors=1) # call fit method Train estimator.fit(x, y) # Data prediction ret = estimator.predict
([[2]]) print(ret) ret = estimator.predict([[30]]) print(ret)
<>K Value selection
<>k Nearest neighbor search algorithm
<>KD tree
<>why
<>what
<>how
<> Establishment of tree
* Multiple dimensions , Select the most dispersed dimension for sorting , Take the median for the first division （ Choose here x axis ）
* Divide left and right according to the median of the dimension selected for the first time （2,4,5 | 8,9）, Sort the corresponding value of another dimension , Median Division
* Repeat the previous two steps , Until it cannot be divided
<> Recent domain search
<> example ： check （2.1,3.1）
<> example ： check （2,4.5）
<>KD Tree summary
<>scikit-learn data set
<>sklearn Small data set
<>sklearn Big data set
<>sklearn Introduction to data set return value
from sklearn.datasets import load_iris,fetch_20newsgroups # Small data set acquisition iris =
load_iris() # print(iris) # Big data set acquisition # news = fetch_20newsgroups() # print(news)
# Dataset attribute description print(" The characteristic value of the dataset is ：\n", iris.data) print(" The target value of the dataset is ：\n", iris["target"])
print(" Data set characteristic value name is ：\n", iris.feature_names) print(" The name of the data set target value is ：\n", iris.
target_names) print(" Dataset description is ：\n", iris.DESCR)
<> Data visualization
from sklearn.datasets import load_iris, fetch_20newsgroups import seaborn as
snsimport matplotlib.pyplot as plt import pandas as pd from pylab import mpl #
Set the display Chinese font mpl.rcParams['font.sans-serif'] = ["SimHei"] # Set normal display symbol mpl.rcParams[
'axes.unicode_minus'] = False # Small data set acquisition iris = load_iris() # Data visualization iris_d = pd.
DataFrame(data=iris.data, columns=["Sepal_Length", "Sepal_Width", "Petal_length"
, "Petal_Width"]) print(iris_d) iris_d["target"] = iris.target def iris_plot(
data, col1, col2): sns.lmplot(x=col1, y=col2, data=data,hue="target",fit_reg=
False) plt.xlabel(col1) plt.ylabel(col2) plt.title(" Species distribution map ") plt.show() iris_plot
(iris_d, "Sepal_Width", "Petal_Length")
<> Data set division
from sklearn.datasets import load_iris, fetch_20newsgroups from sklearn.
model_selectionimport train_test_split x_train, x_test, y_train, y_test =
train_test_split(iris.data, iris.target, random_state=22, test_size=0.2) print(
" The eigenvalue of the training set is ：\n", x_train) print(" The target value of the training set is ：\n", y_train) print(" The characteristic value of the test set is ：\n",
x_test) print(" The target value of the test set is ：\n", y_test) x_train1, x_test1, y_train1, y_test1 =
train_test_split(iris.data, iris.target, random_state=2, test_size=0.2) print(
" The target value of the test set is ：\n", y_test) print(" Test set 1 The target value of is ：\n", y_test1)
<> Feature preprocessing
<> normalization
<> formula
<>api
<> Standardization
<> formula
<>api
<> Pretreatment summary
<> case ： Prediction of Iris species
<> Data set introduction
# @Author : CG # @File : 03- Prediction of Iris species .py # @Time : 2022/2/8 17:19 # @contact:
[email protected] from sklearn.datasets import load_iris from sklearn.
model_selectionimport train_test_split from sklearn.preprocessing import
StandardScalerfrom sklearn.neighbors import KNeighborsClassifier # 1. Get dataset iris
= load_iris() # 2. Basic data processing different random_state It will lead to the difference between the training set and the test set , Which leads to different final accuracy x_train,
x_test, y_train, y_test = train_test_split(iris.data, iris.target, test_size=0.2
, random_state=22) # 3. Characteristic Engineering transfer = StandardScaler() x_train = transfer.
fit_transform(x_train) x_test = transfer.transform(x_test) #fit():
Used to calculate mean（ mean value ） and std（ standard deviation ）, In order to standardize the data later #transform(): according to fit() Function computed mean and std Standardize data
#fit_transform(): yes fit() Functions and transform() Combination of functions , Proceed first fit, Later transform（ Standardization ） #
fit_transform The method is fit and transform Combination of , # fit_transform(X_train)
It means to find out X_train Mean and standard deviation of , And applied in X_train upper . # At this time, for X_test, We can use it directly transform method . #
Because at this time StandardScaler Already saved X_train Mean and standard deviation of .
# Because we have to promise , When the test set is standardized , Unified scaling parameters are used , Is the mean and standard deviation . So use it first fit_transform() On the training set , Reuse transform() On the test set .
# 4. machine learning （ model training ）knn # Instantiation estimator estimator = KNeighborsClassifier(n_neighbors=5) #
model training estimator.fit(x_train, y_train) # 5. Model evaluation # Predicted value result output y_pre = estimator.
predict(x_test) print(" The predicted value is ：\n", y_pre) print(" The difference between the predicted value and the real value is ：\n", y_pre == y_test)
# Accuracy calculation score = estimator.score(x_test, y_test) print(score)
<>KNN Algorithm summary
<> advantage
<> shortcoming
<> Cross validation , Grid search
<> What is cross validation
<> What is grid search
<> Cross validation , Grid search （ Model selection and tuning ）API
# @Author : CG # @File : 03- Prediction of Iris species .py # @Time : 2022/2/8 17:19 # @contact:
[email protected] from sklearn.datasets import load_iris from sklearn.
model_selectionimport train_test_split, GridSearchCV from sklearn.preprocessing
import StandardScaler from sklearn.neighbors import KNeighborsClassifier #
1. Get dataset iris = load_iris() # 2. Basic data processing x_train, x_test, y_train, y_test =
train_test_split(iris.data, iris.target, test_size=0.2, random_state=22) #
3. Characteristic Engineering transfer = StandardScaler() x_train = transfer.fit_transform(x_train)
x_test= transfer.transform(x_test) #fit(): Used to calculate mean（ mean value ） and std（ standard deviation ）, In order to standardize the data later
#transform(): according to fit() Function computed mean and std Standardize data #fit_transform():
yes fit() Functions and transform() Combination of functions , Proceed first fit, Later transform（ Standardization ） #
fit_transform The method is fit and transform Combination of , # fit_transform(X_train)
It means to find out X_train Mean and standard deviation of , And applied in X_train upper . # At this time, for X_test, We can use it directly transform method . #
Because at this time StandardScaler Already saved X_train Mean and standard deviation of .
# Because we have to promise , When the test set is standardized , Unified scaling parameters are used , Is the mean and standard deviation . So use it first fit_transform() On the training set , Reuse transform() On the test set .
# 4. machine learning （ model training ）knn # Instantiation estimator estimator = KNeighborsClassifier(n_neighbors=5) #
Model tuning - Cross validation , Grid search param_grid = {"n_neighbors": [1, 3, 5, 7]} estimator =
GridSearchCV(estimator, param_grid=param_grid, cv=5) # model training estimator.fit(
x_train, y_train) # 5. Model evaluation # Predicted value result output y_pre = estimator.predict(x_test) print(
" The predicted value is ：\n", y_pre) print(" The difference between the predicted value and the real value is ：\n", y_pre == y_test) # Accuracy calculation score =
estimator.score(x_test, y_test) print(score) # Cross validation , Grid search results print(
" Cross validation , The best result of grid search is ：\n", estimator.best_score_) print(" Cross validation , The best model for grid search is ：\n",
estimator.best_estimator_) print(" Cross validation , The model result of grid search is ：\n", estimator.cv_results_)
<> Common distance formula
<> Euclidean distance
<> Manhattan distance
<> Chebyshev Distance
<> Minkowski distance
<> Summary of common distances
<> Other distance formulas
<> Normalized Euclidian Distance
<> Cosine distance
<> Hamming distance
<> Jacquard distance
<> mahalanobis distance
Technology