通过自己不断测试完成这篇:自动将数据从AWS S3同步到Workdocs
https://blog.51cto.com/helpdesk/6145800
但是反向的从Workdocs同步数据到S3就没那么简单了,花了几周时间都无法完成,这个AWS blog上有些地方非常迷惑人:https://docs.aws.amazon.com/workdocs/latest/developerguide/manage-notifications.html
后来还是通过AWS Support,跟2个AWS 工程师一起debug才搞定,踩了不少坑。
主要步骤:
1.- 创建workdocs文件夹,获取文件夹id,创建S3文件夹
2.- 创建Parameter Store,存放workdocs文件夹名和id的对应关系,同时可设定文件格式过滤
3.- 创建SQS
4.- 创建workdocs-to-s3-role
5.- 创建API Gateway
6.- 创建Lambda,关键是自制python requests library,并添加为layer
7.- 设定Workdocs notification
坑:
第2步 Parameter Store, 刚开始没有留意,直接copy文章上的字,例如:{“file_ext”:”.pdf,.xlsx,.csv”}
这个看起来不错,如果用json格式验证会提示双引号问题,如下
第4步创建 workdocs-to-s3-role, Trust relationshi如下:
Permissions如下:
第6步,Lambda layer的坑,之前做S3到Workdocs自动同步到时候遇到过,参考:https://blog.51cto.com/helpdesk/6145800)
第7步,最大的坑,我们的AWS账户使用Organization管理,我的属于子账户,因此需要额外使用assume role才能通过命令行订阅Workdocs的notification
例如:
aws workdocs create-notification-subscription --organization-id d-90xxxxxxxxxxxx --protocol HTTPS --subscription-type ALL --notification-endpoint https://xxxxxxxxx.execute-api.us-east-1.amazonaws.com/dev
An error occurred (UnauthorizedResourceAccessException) when calling the CreateNotificationSubscription operation: UnauthorizedResourceAccessException
这时候只得使用assume role来实现了,具体参考:https://blog.51cto.com/helpdesk/6169003
a1.-创建workdocs_notification的policy, 具体如下:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"workdocs:DeleteNotificationSubscription",
"workdocs:DescribeNotificationSubscriptions",
"workdocs:CreateNotificationSubscription"
],
"Resource": "*"
}
]
}
a2.- 创建workdocs_notification_role, 具体Trust Policy(即Trust relationship), 如下:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::123xxxxxxxxx:root"
},
"Action": "sts:AssumeRole",
"Condition": {}
}
]
}
a3.- 然后将policy附加上去,如下:
a4.- assume role: aws sts assume-role --role-arn arn:aws:iam::123xxxxxxxxxx:role/workdocs_notification_role --role-session-name workdocs-to-s3
获取role对应的keys
然后使用export
export AWS_ACCESS_KEY_ID=ASIAUFSxxxxxxXXXXX
export AWS_SECRET_ACCESS_KEY=AG0YTjpJSejxxxxxxxxxxxxx
export AWS_SESSION_TOKEN=xxxxxxxxxxxxxxxxx
然后:aws sts get-caller-identity
可见已成功assume role权限,如下:
a5.- 进入workdocs,添加notification,这里如果没预先添加,a6步骤是会提示:when calling the CreateNotificationSubscription operation: UnauthorizedResourceAccessException
将workdocs_notification_role的ARN添加进去
a6.- 创建workdocs notification的subscription:aws workdocs create-notification-subscription --organization-id d-9xxxxxxxx --protocol HTTPS --subscription-type ALL --notification-endpoint https://xxxxxxxxxx.execute-api.us-east-1.amazonaws.com/dev
如何查找Workdocs Organization ID: AWS Directory Service