0
点赞
收藏
分享

微信扫一扫

车联网平台部署与验证​

晚熟的猫 2022-05-16 阅读 58
elasticsearchhadoopvimHadoop大数据

车联网平台部署与验证​


  1. 部署Apache Tika​sudo bin/elasticsearch-plugin install ingest-attachment​
    重启es​
  2. elasticsearch部署​10.3.104.139 cq-test-dtp-store-10-3-104-139
    vi /etc/sysctl.conf 添加下面配置： vm.max_map_count=655360 并执行命令(配置生效)： sysctl -p 然后重新启动 elasticsearch即可启动成功。​
    （1）新增es操作账户​
    useradd elasticsearch​
    passwd elasticsearch​
    chown -R elasticsearch:elasticsearch /data/es/elasticsearch​
    （2）修改配置文件​
    su elasticsearch​
    cd /data/es/elasticsearch​
    mkdir data​
    cd config/​
    cp elasticsearch.yml elasticsearch.yml.bak​
    #其余全部注释，直接复制进去，ip根据本机进行修改​
    vi elasticsearch.yml​
    discovery.seed_hosts: ["10.3.104.139"]​
    cluster.initial_master_nodes: ["node-1"] ​
    cluster.name: es ​
    node.name: node-1 ​
    path.data: /data/es/elasticsearch/data ​
    path.logs: /data/es/elasticsearch/logs ​
    network.host: 0.0.0.0 ​
    http.port: 9200 ​
    http.cors.enabled: true​
    http.cors.allow-origin: "*"​
    修改java变量​
    vim /etc/profile​
    export JAVA_HOME=/data/es/elasticsearch/jdk​
    export JRE_HOME=${JAVA_HOME}/jre​
    export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib​
    export PATH=${JAVA_HOME}/bin:$PATH​
    source /etc/profile​
    （3）启动​
    ./elasticsearch -d &​
    访问http://10.3.104.139:9200/​
    #Word文档导入#车联网平台部署与验证​_vim
  3. 部署Cerebro ​# 下载 wget https://github.com/lmenezes/cerebro/releases/download/v0.9.4/cerebro-0.9.4.zip ​
    # 解压 unzip cerebro-0.9.4.zip ​
    # 软链 ln -s cerebro-0.9.4 cerebro​
    修改配置​
    vim conf/application.conf​
    # 这里仅配置hosts ​
    hosts = [ { host = "http://10.3.104.139:9200" name = "ES Cluster" }​
    #Word文档导入#车联网平台部署与验证​_vim_02
    启动​
    bin/cerebro -Dhttp.port=1234 -Dhttp.address=10.3.104.139
    ​​​http://10.3.104.139:1234/#!/overview?host=http:%2F%2F10.3.104.139:9200%2F​​​
  4. 部署kibana ​#压缩部署​
    curl -O https://artifacts.elastic.co/downloads/kibana/kibana-7.6.2-linux-x86_64.tar.gz​
    tar -xzf kibana-7.6.2-linux-x86_64.tar.gz​
    cd kibana-7.6.2-linux-x86_64/​
    #修改配置文件​
    vim /data/kibana-7.6.2-linux-x86_64/config/kibana.yml ​
    server.host: "0.0.0.0"​
    I18n.locale: "zh-CN"​
    #启动​
    [root@cq-test-dtp-store-10-3-104-139 data]# chown -R elasticsearch:elasticsearch kibana-7.6.2-linux-x86_64​
    [root@cq-test-dtp-store-10-3-104-139 data]# su elasticsearch​
    ./bin/kibana &​
    ​​​http://10.3.104.139:5601/app/kibana​​​
  5. 部署Minio#下载
    wget https://dl.minio.io/server/minio/release/linux-amd64/minio
    chmod +x minio
    ./minio server /data
  1. 数据存储目录​

[root@cq-test-dtp-store-10-3-104-190 data]# mkdir /data/data1​

[root@cq-test-dtp-store-10-3-104-190 data]# mkdir /data/data2​

[root@cq-test-dtp-store-10-3-104-190 data]# mkdir /data/data3​

[root@cq-test-dtp-store-10-3-104-190 data]# mkdir /data/data4​

启动脚本目录​

mkdir -p /data/minio​

  1. 集群配置文件目录​

mkdir -p /etc/minio​

  1. 编写集群启动脚本（所有节点配置文件相同）​

vim /data/minio/run.sh​

#!/bin/bash​

export MINIO_ACCESS_KEY=miniotest​

export MINIO_SECRET_KEY=miniotest​

/data/minio/minio server --config-dir /etc/minio \​

http://10.3.104.140/minio/data1 http://10.3.104.140/minio/data2 \​

http://10.3.104.143/minio/data1 http://10.3.104.143/minio/data2 \​

http://10.3.104.183/minio/data1 http://10.3.104.183/minio/data2 \​

http://10.3.104.190/minio/data1 http://10.3.104.190/minio/data2 > /opt/minio.log​

  1. 编写服务脚本（所有节点）​

vim /usr/lib/systemd/system/minio.service​

[Unit]​

Description=Minio service​

Documentation=https://docs.minio.io/​

[Service]​

WorkingDirectory=/data/minio/​

ExecStart=/data/minio/run.sh​

Restart=on-failure​

RestartSec=5​

[Install]​

WantedBy=multi-user.target​

chmod +x /usr/lib/systemd/system/minio.service​

将minio上传到/data/minio目录下并赋予权限​

chmod +x minio​

启动​

systemctl daemon-reload​

systemctl start minio​

systemctl enable minio​

systemctl status minio.service​

#密码启动​

export MINIO_ACCESS_KEY=miniotest​

export MINIO_SECRET_KEY=miniotest​

​​http://10.3.104.190:37713/dashboard​​​

  1. 部署ozone

创建目录​

创建Datanode的数据目录 挂载到磁盘上​

# mkdir -p /data/disk{1..2}​

# chown -R hadoop:hadoop /data/disk{1..2}​

创建ozone相关目录​

# mkdir -p /data/bigdata/{conf,soft,logs,metadata,tmp}​

# chown -R hadoop:hadoop /data/bigdata​

$ mkdir -p /data/bigdata/logs/ozone​

$ mkdir -p /data/bigdata/tmp/run​

$ mkdir -p /data/bigdata/metadata/ozone/{om,node,recon,ratis,scm}​

环境变量​

所有节点都需配置​

配置ozone​

export OZONE_HOME=/data/bigdata/soft/ozone​

export PATH=$OZONE_HOME/sbin:$OZONE_HOME/bin:$PATH​

export HADOOP_dataS="-XX:ParallelGCThreads=8 -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+CMSParallelRemarkEnabled"​

export HADOOP_PID_DIR=/data/bigdata/tmp/run​

export HADOOP_LOG_DIR=/data/bigdata/logs/hadoop​

拷贝软件​

Ozone 下载的二进制包，直接解压使用，拷贝至所有节点​

$ tar -zxvf hadoop-ozone-1.0.0.tar.gz -C /data/bigdata/soft/​

$ cd /data/bigdata/soft/​

$ mv ozone-1.0.0 ozone​

修改配置​

生成配置​

$ ozone genconf $OZONE_HOME/etc/hadoop​

修改配置​

$ vim $OZONE_HOME/etc/hadoop/ozone-site.xml​

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>​

<configuration>​

<configuration>​

<property>​

<name>ozone.om.address</name>​

<value>10.3.104.140</value>​

</property>​

<property>​

<name>ozone.om.http-address</name>​

<value>10.3.104.140:9874</value>​

</property>​

<property>​

<name>ozone.om.db.dirs</name>​

<value>/data/bigdata/metadata/ozone/om</value>​

</property>​

<property>​

<name>ozone.metadata.dirs</name>​

<value>/data/bigdata/metadata/ozone</value>​

</property>​

<property>​

<name>ozone.om.ratis.enable</name>​

<value>false</value>​

</property>​

<property>​

<name>ozone.om.service.ids</name>​

<value>ombigdata001</value>​

</property>​

<property>​

<name>ozone.om.nodes.ombigdata001</name>​

<value>om1,om2,om3</value>​

</property>​

<property>​

<name>ozone.om.address.ombigdata001.om1</name>​

<value>10.3.104.140</value>​

</property>​

<property>​

<name>ozone.om.address.ombigdata001.om2</name>​

<value>10.3.104.143</value>​

</property>​

<property>​

<name>ozone.om.address.ombigdata1.om3</name>​

<value>10.3.104.183</value>​

</property>​

<property>​

<name>ozone.scm.client.address</name>​

<value>10.3.104.140</value>​

</property>​

<property>​

<name>ozone.scm.names</name>​

<value>10.3.104.140</value>​

</property>​

<property>​

<name>ozone.scm.container.size</name>​

<value>5GB</value>​

</property>​

<property>​

<name>ozone.scm.db.dirs</name>​

<value>/data/bigdata/metadata/ozone/scm</value>​

</property>​

<property>​

<name>ozone.scm.pipeline.owner.container.count</name>​

<value>3</value>​

</property>​

<property>​

<name>ozone.scm.datanode.id.dir</name>​

<value>/data/bigdata/metadata/ozone/node</value>​

</property>​

<property>​

<name>ozone.scm.block.client.address</name>​

<value>10.3.104.140</value>​

</property>​

<property>​

<name>ozone.scm.datanode.address</name>​

<value>10.3.104.140</value>​

</property>​

<property>​

<name>hdds.datanode.dir</name>​

<value>/data/disk1,/data/disk2</value>​

</property>​

<property>​

<name>ozone.recon.db.dir</name>​

<value>/data/bigdata/metadata/ozone/recon</value>​

</property>​

<property>​

<name>ozone.recon.address</name>​

<value>10.3.104.140:9891</value>​

</property>​

<property>​

<name>recon.om.snapshot.task.interval.delay</name>​

<value>1m</value>​

</property>​

<property>​

<name>dfs.container.ratis.datanode.storage.dir</name>​

<value>/data/bigdata/metadata/ozone/ratis</value>​

</property>​

</configuration>​

$ vim $OZONE_HOME/etc/hadoop/workers​

10.3.104.143​

10.3.104.140​

10.3.104.190​

10.3.104.183​

启动服务​

scm​

$ ozone scm --init​

$ ozone --daemon start scm​

http://10.3.104.140:9874​

om​

$ ozone om --init​

$ ozone --daemon start om​

http://10.3.104.140:9876/#!/​

#Word文档导入#车联网平台部署与验证​_elasticsearch_03

datanode –所有datanode节点​

$ ozone --daemon start datanode​

recon​

$ ozone --daemon start recon​

s3g​

$ ozone --daemon start s3g​

校检验证​

​​http://10.3.104.140:9888/#/Overview​​​

​​http://10.3.104.140:9882/#!/​​​

​​http://10.3.104.140:9878/static/​​​

​​http://10.3.104.140:9876/#!/​​​

  1. 环境部署并验证​
  2. URL​


Ozone:​

​​http://10.3.104.140:9888/#/Overview​​​

​​http://10.3.104.140:9882/#!/​​​

​​http://10.3.104.140:9878/static/​​​

​​http://10.3.104.140:9876/#!/​​.​

Es:​

​​http://10.3.104.139:9200/​​​

Cerebro:​

​​http://10.3.104.139:1234/#!/overview?host=http:%2F%2F10.3.104.139:9200%2F​​​

Kibanna:​

​​http://10.3.104.139:5601/app/kibana​​​

Minio​

​​http://10.3.104.190:37713/dashboard​​​



举报

相关推荐

车联网平台如何提升安全性能浅谈

分布式

车联网简介

自动驾驶人工智能计算机视觉车联网5g

车联网 TSP 平台场景中的 MQTT 主题设计

车联网mqttemq物联网iot嵌入式

车联网平台百万级消息吞吐架构设计

车联网物联网iotmqttemq嵌入式

车联网平台数据概括数据可视化

数据车联网数据可视化人工智能

上平台！ 车联网智能化晋级高段位！

车联网数据云平台物联网嵌入式

【车联网安全】车端知识调研

安全

什么是车联网

车联网车联网介绍物联网大数据JavaScript前端开发

极氪与 TDengine 合作建设新一代车联网云端基础平台

tdengine大数据人工智能时序数据库数据库

危化车安全监测远程管理物联网平台

数据实时视频安全防护物联网
0 条评论