Deploy kubernetes and helm (国内部署 kubernetes)

国内部署 k8s 是真的难，坑太多了。这两个花了两天终于成功部署一个能用都了

尝试

首先，我从alpine系统下尝试部署，可是源里面都 test kubernetes 不能用。运行 kubeadm init 每次都是部署失败，即使kubelet启动成功了。但是卡在docker这里，因为其他组件都不会成功运行。日志也没有任何相关资料，github上都问题解决方案都没有用。所以，必须得换一个系统。 并且国内墙了google，源都不能访问。

过程

我选用了Debian 10作为我都主系统。

Docker

安装完成后，可以直接通过 curl https://get.docker.com/ | bash - 来安装docker。 不知道为什么 Debian 10 安装不上 cri-o。docker 用起来也没问题，所以就不在这里纠结了。 然后检查一下运行状况，没问题，修改一下配置。

{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "insecure-registries" : [
    "10.0.1.121:5000",
    "127.0.0.1:5000"
  ],
  "registry-mirrors": ["https://************.myhuaweicloud.com"],
  "storage-driver": "overlay2",
  "data-root": "/mnt/docker"
}

这里我配置了华为的镜像，需要先在这里注册，就能获取一个链接。kubeadm 推荐systemd 作为 cgroup 的驱动，所以改动一下。

Kubernetes

之后就可以开始安装 k8s了。 参考了一下这些文章：

1. debian9.8搭建kubernetes多主节点集群(一)
2. kubernetes遇到问题
3. k8s 1.12.1 的坑和解决方法
4. Kubernetes使用中发现的问题和错误
5. Creating a single control-plane cluster with kubeadm
6. Kubernetes on bare-metal in 10 minutes

按照第一篇文章「木子甘」，先进行了系统的设置

echo -e "net.bridge.bridge-nf-call-ip6tables =1\nnet.bridge.bridge-nf-call-iptables =1\nnet.ipv4.ip_forward = 1" >> /etc/sysctl.conf;sysctl -p;swapoff -a;sed -ri "/swap/s@(.*)@#/&@g" /etc/fstab;echo "SELINUX=disabled" > /etc/selinux/config

然后加入阿里云的源来按照k8s的主要程序。

apt-get install -y apt-transport-https curl && \
curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg |apt-key add -  && \
echo "deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main"  >>/etc/apt/sources.list.d/kubernetes.list  && \
apt update

之后就可以固定住他们的版本了，这样能保证稳定性。

apt-mark hold kubelet kubeadm kubectl docker-ce

在配置开始之前，先获取下docker的镜像：

IMAGE="kube-apiserver:v1.15.2 kube-controller-manager:v1.15.2 kube-scheduler:v1.15.2 kube-proxy:v1.15.2 pause:3.1 etcd:3.3.10 coredns:1.3.1"
for imageName in ${IMAGE} ; do
    docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName
    
    docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName k8s.gcr.io/$imageName
done

这里从其他源下载了镜像，然后改成k8s的名字。注意版本要按照刚刚安装的版本一样，之后就可以开始配置k8s了。

然后就可以通过kubeadm进行初始化了。这里设置了内网的ip，还有api服务器的ip。

kubeadm init --pod-network-cidr=10.0.1.0/16 --apiserver-advertise-address=10.0.1.***

之后，如果没有报错k8s就算成功运行了。

配置账号

如果运行成功，他会提示你配置一下你的账号。要不然不能管理服务。按照提示输入就好了。

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

配置网络

在这k8s成功运行但是他还没准备好。因为网络策略还没配置。详细的配置程序在官方文档有很详细的说明。我挑选了calico。

kubectl apply -f https://docs.projectcalico.org/v3.8/manifests/calico.yaml

至此，k8s就算准备好了。但是它默认禁止了主节点不会运行其他的程序，所以我们要先把那个去掉

kubectl get no -o yaml | grep taint -A 5
kubectl taint nodes --all node-role.kubernetes.io/master-

这样就大功告成了

Helm

参考了这位大大写的kunernets使用helm安装tiller踩坑。真的解决了我的问题。 Helm安装很简单，直接从github上的release会有各个版本的编译文件。直接下载下来放在/usr/local/bin就好。 之后就可以进行配置了。首先，如果打开了k8s的rbac，那么我们得先建立一个账号。

apiVersion: v1
kind: ServiceAccount
metadata:
  name: tiller
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: tiller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - kind: ServiceAccount
    name: tiller
    namespace: kube-system

写入到rbac-config.yaml就可以用命令导入账号了。

$ kubectl create -f rbac-config.yaml
serviceaccount "tiller" created
clusterrolebinding "tiller" created
$ helm init --service-account tiller --history-max 200

但是这里也是得先获得一个tiller的镜像。我在docker hub上面找到了doublemine/kubernetes-helm.tiller的镜像。 然后可以参照这个文档来签名。然后带上一个helm的源就可以开始配置了。我们还是选择阿里云的源。

helm init --debug --tiller-tls --tiller-tls-cert ./tiller.cert.pem --tiller-tls-key ./tiller.key.pem --tiller-tls-verify --tls-ca-cert ca.cert.pem --tiller-image doublemine/kubernetes-helm.tiller:v2.14.3 --stable-repo-url https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts --service-account tiller

这里安装就完成了。

其他可能有用的

1. 构建单节点的master的k8s应用 集成helm管理(参考了一些其他大神的资料)
2. Kubernetes 集群使用 Helm 搭建 GitLab 并配置 Ingress

Spilt traffic using route table in mac

When working remotely, we need to access internal resource. Due to security concern, that they are often restricted to only access from local. Then, we need to connect to vpn that enable us to interact with internal server.

However, we do not want to send all the request over vpn, since it might be slow. How do we control how traffic over vpn?

This can be achive by using route command.

1. Connect to the vpn, uncheck send all traffic over VPN connection
2. Delete route to vpn by default. sudo route delete -net default -interface ppp0
3. Add route to physical interface. sudo route add -net 0.0.0.0 -interface en0
4. Add ip range route to vpn. sudo route add -net 10.0.1.0 -netmask 255.255.255.0 -interface ppp0

Then you are done, and only this ip with in the range will send via vpn. Source: link

Install Oracle Java from source

Background

I am using jhipster to generate spring boot project. It compile but it does not work well. The first thing is that some function is not compatible. Also, hibernate insert data twice that cause ConstraintViolationException. I had post a Issue.

It works well on Java 8, so I have to install one more version. However, I cannot install via PPA. It is because webupd8 has DISCONTINUED since we need to login to download. So, I have to download java manually. I can use update-alternatives to switch version.

Install

1. Download 1.8 jdk from Oracle. Register an account and then download. It is better to download jdk-8u211-linux-x64.tar.gz.
2. scp upload the file to server.
3. unzip the file at server with tar xzvf filename.
4. Move the folder into /usr/lib/jvm/ (The location my java 11 was installed).
5. Then use update-alternatives with commands.

   update-alternatives --install /usr/bin/java java /usr/lib/jvm/java-8-oracle-amd64/bin/java 100
   update-alternatives --install /usr/bin/javac javac /usr/lib/jvm/java-8-oracle-amd64/bin/javac 100
   update-alternatives --install /usr/bin/jar jar /usr/lib/jvm/java-8-oracle-amd64/bin/jar 100
   

6. Update command to specify version.

   update-alternatives --config java
   update-alternatives --config javac
   update-alternatives --config jar
   

7. Check java version if it works.
8. write $JAVA_HOME to specify location. I write it to /usr/lib/jvm/default-java and link to directory with ln.

Then when you switch version, you just change the link to specify version. Inspired by Rahul.

cUrl Test in local

So, when we try to access our local development website, we tend to access the website via http://localhost:8080/. However, when we need to test for domain name functionality, we need to manually resolve it in hosts.

With cURL, we can easily resolve the domain manually. Use --resolve HOST:PORT:ADDRESS Force resolve of HOST:PORT to ADDRESS, we can change the address of the host.

# access http
curl --resolve 'mai1015.com:80:127.0.0.1' http://mai1015.com/path
# access http
curl --resolve 'mai1015.com:443:127.0.0.1' https://mai1015.com/path

So, the port is included in part of the resolve.

Or we can use -H, --header LINE Pass custom header LINE to server (H) to hack the header. We can change the host in the header. And, the server assumes it is accessing the specified server.

curl -H 'Host: mai1015.com' http://127.0.0.1/something

However, it does not work in https because the certificate domain does not match the URL. It is better to use --resolve.

Source: stackoverflow

Javascript open new Window blocked when asynchronous

Problem

When you trying to open a new tag for your user. It is ok if we just use window.open right away. However, we need to do complex task like making request from server to generate the link. Then it will be block by the browser.

Solution

The solution is pretty straight forward. You can open new tab and modify it after you get the link. This is the sample code below.

async clicked(e) {
    // create new window
    var w = window.open('', '_blank');
    // fetch url
    var url = await someFunction();
    // then modify the location
    w.location.href = w;
}

The reason of that is the tab should be open during that user click event loop. Otherwise, it is going to assume that is a pop-up that should be blocked. That is why you can get ad pop-up on some site when you click.

This is from stackoverflow