-
Notifications
You must be signed in to change notification settings - Fork 16
/
Copy pathmigrate-atmos-s3-to-ecs-s3.html.md.erb
120 lines (94 loc) · 4.54 KB
/
migrate-atmos-s3-to-ecs-s3.html.md.erb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
---
title: Migrate from legacy ATMOS S3 to EMC ECS S3
owner: Services
---
This guide teaches you how to migrate from the deprecated ATMOS S3 (dynstrg) to the new EMC ECS S3 (dynstrg-2).
Locate an instance of the old Atmos S3 Service (dynstrg) in your space:
<pre class="terminal">
$ cf services
Getting services in org INI-DOS-DPS-TCL_user / space playground as USER03...
name service plan bound apps last operation
holiday-pictures dynstrg usage create succeeded
</pre>
Check the marketplace to see the new S3 service:
<pre class="terminal">
$ cf marketplace | grep dynstrg
dynstrg usage* Dynamic Object Storage with S3 Interface based on ATMOS v2.2.3
dynstrg-2 usage* Dynamic Object Storage with S3 API based on EMC ECS 3.0
</pre>
<strong>Migrate your Buckets to the new service:</strong>
*Important*: Please, make sure that you have the latest [cf CLI](https://github.com/cloudfoundry/cli) installed before you follow this migration guide.
First, create a new dynstrg-2 service in the same space, let's call it "new-holiday-pictures":
<pre class="terminal">
$ cf create-service dynstrg-2 usage new-holiday-pictures
Creating service instance new-holiday-pictures in org INI-DOS-DPS-TCL / space playground
</pre>
Next, stop all your apps which use dynstrg so that no new objects are stored during the migration:
<pre class="terminal">
$ cf stop my-awesome-app
</pre>
As a next step, you'll have to download the minio S3 client. You will need it to copy one bucket after another recursively from the old service to the new one.
For all operating systems, visit the minio website [minio](https://www.minio.io/downloads.html#download-client)
To enable access to both services for your minio client, get the credentials by generating a service key on each service:
<pre class="terminal">
cf create-service-key holiday-pictures mys3servicekey
cf create-service-key new-holiday-pictures mys3servicekey
</pre>
Run this command to get the credentials:
<pre class="terminal">
cf service-key holiday-pictures mys3servicekey
{
"accessHost": "ds11s3.swisscom.com",
"accessKey": "5484335407854yolo7dc88e01206fc148/I-AM-JUST-AN-EXAMPLE-KEY",
"sharedSecret": "<redacted>"
}
</pre>
Open the minio config (.mc/config-json) and enter the credentials in the following way:
<strong>*Important*: Please make sure that you use https and set the right api version!</strong>
<pre class="terminal">
{
"version": "9",
"hosts": {
"holiday": {
"url": "https://ds11s3.swisscom.com",
"accessKey": "5484335407854yolo7dc88e01206fc148/OF-COURSE-I-STILL-LOVE-YOU",
"secretKey": "<redacted>",
"api": "S3v2",
"lookup": "auto"
},
"holiday-dynstrg2": {
"url": "https://ds11s3.swisscom.com",
"accessKey": "85d9b397330nisi64920a1409f4732119-JUST-READ-THE-INSTRUCTIONS",
"secretKey": "<redacted>",
"api": "S3v4",
"lookup": "auto"
}
}
</pre>
Before you can use the mc command (minio) to mirror a bucket, you need to create an empty bucket on the new service. In this example you have two buckets in your old service, "pictures" and "tickets":
<pre class="terminal">
mc mb holiday-dynstrg2/tickets
Bucket created successfully `holiday-dynstrg2/tickets`.
mc mb holiday-dynstrg2/pictures
Bucket created successfully `holiday-dynstrg2/pictures`.
</pre>
Finally, you are able to migrate all the buckets from the old to the new service:
<pre class="terminal">
mc mirror holiday/tickets holiday-dynstrg2/tickets
...bucket/ticket_japan.jpg: 18 B / 18 B
</pre>
To verify that everything succeeded, list the new bucket:
<pre class="terminal">
mc ls holiday-dynstrg2/tickets
[2018-07-05 16:22:57 UTC] 6B ticket_malaysia.jpg
[2018-07-05 16:22:57 UTC] 6B ticket_japan.jpg
[2018-07-05 16:22:57 UTC] 6B ticket_china.jpg
</pre>
<strong>Repeat this procedure for every S3 service and bucket you own!</strong>
As soon as everything is working properly, you can remove the service keys and the old service:
<pre class="terminal">
$ cf delete-service-key holiday-pictures mys3servicekey
$ cf delete-service-key new-holiday-pictures mys3servicekey
$ cf delete-service holiday-pictures
</pre>
You now migrated your buckets to the new S3 service. Congrats! You can now bind your apps to the new service and start them again.