maxwell监听mysql的新增和修改操作触发事件

http://maxwells-daemon.io/quickstart/

Download

• Download binary distro:​​https://github.com/zendesk/maxwell/releases/download/v1.33.0/maxwell-1.33.0.tar.gz​​
• Sources and bug tracking is available on github:​​https://github.com/zendesk/maxwell​​

curl:

curl -sLo - https://github.com/zendesk/maxwell/releases/download/v1.33.0/maxwell-1.33.0.tar.gz \
       | tar zxvf -
cd maxwell-1.33.0

docker:

docker pull zendesk/maxwell

homebrew:

brew install maxwell

Configure Mysql

Server Config: Ensure server_id is set, and that row-based replication is on.

$ vi my.cnf

[mysqld]
server_id=1
log-bin=master
binlog_format=row

Or on a running server:

mysql> set global binlog_format=ROW;
mysql> set global binlog_row_image=FULL;

note: ​​binlog_format​​ is a session-based property. You will need to shutdown all active connections to fully convert to row-based replication.

Permissions: Maxwell needs permissions to act as a replica, and to write to the ​​maxwell​​ database.

mysql> CREATE USER 'maxwell'@'%' IDENTIFIED BY 'XXXXXX';
mysql> GRANT ALL ON maxwell.* TO 'maxwell'@'%';
mysql> GRANT SELECT, REPLICATION CLIENT, REPLICATION SLAVE ON *.* TO 'maxwell'@'%';

# or for running maxwell locally:

mysql> CREATE USER 'maxwell'@'localhost' IDENTIFIED BY 'XXXXXX';
mysql> GRANT ALL ON maxwell.* TO 'maxwell'@'localhost';
mysql> GRANT SELECT, REPLICATION CLIENT, REPLICATION SLAVE ON *.* TO 'maxwell'@'localhost';

Run Maxwell

Command line

bin/maxwell --user='maxwell' --password='XXXXXX' --host='127.0.0.1' --producer=stdout

Docker

docker run -it --rm zendesk/maxwell bin/maxwell --user=$MYSQL_USERNAME \
    --password=$MYSQL_PASSWORD --host=$MYSQL_HOST --producer=stdout

Kafka

Boot kafka as described here: ​​http://kafka.apache.org/documentation.html#quickstart​​, then:

bin/maxwell --user='maxwell' --password='XXXXXX' --host='127.0.0.1' \
   --producer=kafka --kafka.bootstrap.servers=localhost:9092 --kafka_topic=maxwell

(or docker):

docker run -it --rm zendesk/maxwell bin/maxwell --user=$MYSQL_USERNAME \
    --password=$MYSQL_PASSWORD --host=$MYSQL_HOST --producer=kafka \
    --kafka.bootstrap.servers=$KAFKA_HOST:$KAFKA_PORT --kafka_topic=maxwell

Kinesis

docker run -it --rm --name maxwell -v `cd && pwd`/.aws:/root/.aws zendesk/maxwell sh -c 'cp /app/kinesis-producer-library.properties.example \
/app/kinesis-producer-library.properties && echo "Region=$AWS_DEFAULT_REGION" >> /app/kinesis-producer-library.properties && bin/maxwell \
--user=$MYSQL_USERNAME \
--password=$MYSQL_PASSWORD --host=$MYSQL_HOST  \
 --producer=kinesis --kinesis_stream=$KINESIS_STREAM'  \

Nats

bin/maxwell --user='maxwell' --password='XXXXXX' --host='127.0.0.1' \
    --producer=nats --nats_url=='0.0.0.0:4222'

Google Cloud Pub/Sub

bin/maxwell --user='maxwell' --password='XXXXXX' --host='127.0.0.1' \
  --producer=pubsub --pubsub_project_id='$PUBSUB_PROJECT_ID' \
  --pubsub_topic='maxwell'

RabbitMQ

bin/maxwell --user='maxwell' --password='XXXXXX' --host='127.0.0.1' \
    --producer=rabbitmq --rabbitmq_host='rabbitmq.hostname'

Redis

bin/maxwell --user='maxwell' --password='XXXXXX' --host='127.0.0.1' \
    --producer=redis --redis_host=redis.hostname

SNS

bin/maxwell --user='maxwell' --password='XXXXXX' --host='127.0.0.1' \
    --producer=sns --sns_topic=sns.topic --sns_attrs=database,table

启动：

./bin/maxwell --user='root' --password='6677mCom' --host='127.0.0.1'   --producer=rabbitmq --rabbitmq_host='192.168.2.232' --rabbitmq_port='5672' \

 --rabbitmq_routing_key_template="maxwell.%table%" --rabbitmq_exchange='maxwell' --rabbitmq_exchange_type='direct'

创建队列 并绑定路由

{"database":"liexin_crm","table":"lie_invoice_company","type":"update","ts":1618917918,"xid":46607,"commit":true,"data":{"id":28,"company_type":0,"com_sn":"810980","erp_com_sn":"","user_id":1600,"tax_id":0,"com_name":"白云公司123456","com_addr":"ggg","com_tel":"hhh","com_tax_registration":"咕咕咕咕咕咕tt","com_bank":"订单的忍忍tttrrr","com_bank_num":"","com_nature":0,"establishment_time":0,"registered_capital":0.0000,"com_organization":"","legal_representative":"","five_legal_change":0,"two_years_no_lawsuit":"","total_credit_score":0.00,"ichunt_hunting_core":0,"blacklist_hit":"","risk_type":0,"is_kefu_certified":1,"customer_en":"","customer_type":0,"product_use_classone_sn":-1,"product_use_classtwo_sn":-1,"customer_website":"","customer_desc":"","business_license_src":"","sale_department":0,"status":1,"create_time":1601016871,"update_time":1615952875},"old":{"com_addr":"","com_tel":"","com_tax_registration":"咕咕咕咕咕咕"}}