TogoDX server on AWS

やっていくぞ

VCPをつくる (AWSの中に自分のネットワーク環境をつくる)

Internet Gateway をつくって VCP にくっつける (VCPにインターネットからアクセスできるようにする)

Subnet をつくる (EC2とかを置くSubnetをつくる)

複数のAZに1つずつ

1aがALB建てられないエラー出たので1cと1dで

Route table を設定する (つくった Subnet を VCP に紐付ける)

ALBをつくる

VCP を指定する

アクセスをバラす Subnet を複数指定する

ALBにアクセスするための Security group を作って ALB にくっつける

Secutiry group も作るときに VCP にくっつける

残りの設定は後で

EC2をたてる

Amazon Linux 2 !!

public ipv4 address をつけるのを忘れた、後からはつけられないらしい

NVMe SSD 付きの c5d-4xlarge にしたぞ

200GB EBS storage

NVMe SSD をマウントする

code:shell

$ lsblk

NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT

nvme0n1 259:0 0 200G 0 disk

├─nvme0n1p1 259:2 0 200G 0 part /

└─nvme0n1p128 259:3 0 1M 0 part

nvme1n1 259:1 0 372.5G 0 disk

$ sudo file -s /dev/nvme1n1

/dev/nvme1n1: data

$ sudo mkfs -t xfs /dev/nvme1n1

meta-data=/dev/nvme1n1 isize=512 agcount=4, agsize=24414063 blks

= sectsz=512 attr=2, projid32bit=1

= crc=1 finobt=1, sparse=0

data = bsize=4096 blocks=97656250, imaxpct=25

= sunit=0 swidth=0 blks

naming =version 2 bsize=4096 ascii-ci=0 ftype=1

log =internal log bsize=4096 blocks=47683, version=2

= sectsz=512 sunit=0 blks, lazy-count=1

realtime =none extsz=4096 blocks=0, rtextents=0

$ sudo mkdir /data

$ sudo mount /dev/nvme1n1 /data

$ sudo cp /etc/fstab /etc/fstab.bu

$ sudo blkid

(find uuid of the block)

$ sudo nano /etc/fstab

(append a line "UUID=XXXXX / xfs defaults,nofail 0 2")

$ lsblk

NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT

nvme0n1 259:0 0 200G 0 disk

├─nvme0n1p1 259:2 0 200G 0 part /

└─nvme0n1p128 259:3 0 1M 0 part

nvme1n1 259:1 0 372.5G 0 disk /data

$ sudo chmod -R 777 /data

Docker を入れる

code:shell

$ sudo yum update -y

$ sudo amazon-linux-extras install -y docker

$ sudo service docker start

$ sudo usermod -a -G docker ec2-user

$ docker info

Client:

Context: default

Debug Mode: false

Server:

Containers: 0

Running: 0

Paused: 0

Stopped: 0

Images: 0

Server Version: 20.10.7

Storage Driver: overlay2

Backing Filesystem: xfs

Supports d_type: true

Native Overlay Diff: true

userxattr: false

Logging Driver: json-file

Cgroup Driver: cgroupfs

Cgroup Version: 1

Plugins:

Volume: local

Network: bridge host ipvlan macvlan null overlay

Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog

Swarm: inactive

Runtimes: runc io.containerd.runc.v2 io.containerd.runtime.v1.linux

Default Runtime: runc

Init Binary: docker-init

containerd version: d71fcd7d8303cbf684402823e425e9dd2e99285d

runc version: 84113eef6fc27af1b01b3181f31bbaf708715301

init version: de40ad0

Security Options:

seccomp

Profile: default

Kernel Version: 5.10.75-79.358.amzn2.x86_64

Operating System: Amazon Linux 2

OSType: linux

Architecture: x86_64

CPUs: 16

Total Memory: 30.57GiB

Name: ip-10-0-1-58.ap-northeast-1.compute.internal

ID: NQFU:LCYS:OZE4:4FM3:W3ID:EGVY:UBRZ:WYRD:XXPP:KM5F:IDBK:S5RQ

Docker Root Dir: /var/lib/docker

Debug Mode: false

Labels:

Experimental: false

Insecure Registries:

127.0.0.0/8

Live Restore Enabled: false

client が古いっぽくて compose は bundle されてないらしい

先人に感謝

code:shell

$ sudo chmod +x /usr/local/bin/docker-compose

$ docker-compose version

Docker Compose version v2.1.1

必要に応じて色々入れていく

code:shell

$ sudo yum install -y git

$ cd togodx-server

$ mkdir data

$ cd data

$ time tar xf pg_data_idx.tar.gz

tar xf pg_data_idx.tar.gz 539.27s user 126.40s system 56% cpu 19:30.60 total

試しにマウントした NVMe ボリュームの上でやってみる

code:shell

$ cd /data

$ mkdir tmp

$ cd tmp

Resolving ep6.dbcls.jp (ep6.dbcls.jp)... 133.39.78.89

Connecting to ep6.dbcls.jp (ep6.dbcls.jp)|133.39.78.89|:80... connected.

HTTP request sent, awaiting response... 200 OK

Saving to: ‘pg_data_idx.tar.gz’

$ time tar xf pg_data_idx.tar.gz

tar xf pg_data_idx.tar.gz 538.55s user 128.12s system 118% cpu 9:23.89 total

通常のボリュームの上だと 30-40% くらいだった (= I/O が律速になっている)

こんなにパフォーマンス違うのか

じゃDB動かすのもこっちで……

listen_addresses = '*' # what IP address(es) to listen on;

しばらく間が空いたので改めてやっていきます

code:terminal

$ cd

$ mkdir data

$ cd data

$ tar xf pg_data_idx.tar.gz

$ mv postgresql pg_data

$ nano pg_data/postgresql.conf

code:pg_data/postgresql.conf

listen_addresses = '*' # what IP address(es) to listen on;

port = 5432

code:terminal

$ cd ~/togodx-server

$ mv -f config/database.pg.yml.example config/database.yml

$ cat config/database.yml

code:config/database.yml

default: &default

adapter: postgresql

encoding: unicode

pool: <%= ENV.fetch('RAILS_MAX_THREADS') { 5 } %>

host: <%= ENV.fetch('TOGODX_SERVER_DATABASE_HOST') { 'localhost' } %>

port: <%= ENV.fetch('TOGODX_SERVER_DATABASE_PORT') { 5432 } %>

username: <%= ENV.fetch('TOGODX_SERVER_DATABASE_USER') { 'togodx' } %>

development:

<<: *default

database: togodx_server_development

test:

<<: *default

database: togodx_server_test

production:

<<: *default

database: togodx_server_production

code:terminal

$ mv .env.example .env

$ cat .env

code:.env

# Rails

TOGODX_SERVER_DATABASE_HOST=db

TOGODX_SERVER_DATABASE_USER=togodx

TOGODX_SERVER_DATABASE_PASSWORD=changeme

# PostgreSQL

POSTGRES_USER=togodx

POSTGRES_PASSWORD=changeme

# Nginx

NGINX_PORT=80

code:terminal

$ docker-compose -f docker-compose.dev.yml up -d

$ docker-compose -f docker-compose.dev.yml exec db bash

# psql -h localhost -p 5432 -d togodx_server_development -U togodx

psql (14.1 (Debian 14.1-1.pgdg110+1))

Type "help" for help.

togodx_server_development=# \du

List of roles

Role name | Attributes | Member of

-----------+------------------------------------------------------------+-----------

togodx | Superuser, Create role, Create DB, Replication, Bypass RLS | {}

togodx_server_development=# ALTER USER togodx WITH PASSWORD 'new_password';

ALTER ROLE

togodx_server_development=# \q

$ curl localhost:3000/breakdown/gene_chromosome_ensembl

次は production でやる

これが結構めんどい

EDITOR が開くのでinteractiveにやる必要があるらしい

秘匿情報を扱っているわけではないのに……

code:terminal

$ cd ~/togodx-dx-server

$ docker run --rm -it -v $(pwd):/work -w /work ruby:3.0.2 bash

# bundle install

# apt update -y && apt install -y nano

# EDITOR=nano bundle exec rails credentials:edit

(save the credential on editor)

File encrypted and saved.

# ls config

application.rb cable.yml database.sqlite3.yml.example environment.rb initializers master.key routes.rb storage.yml

boot.rb credentials.yml.enc database.yml environments locales puma.rb spring.rb togodx.yml

# exit

code:config/database.yml

default: &default

adapter: postgresql

encoding: unicode

pool: <%= ENV.fetch('RAILS_MAX_THREADS') { 5 } %>

host: <%= ENV.fetch('TOGODX_SERVER_DATABASE_HOST') { 'localhost' } %>

port: <%= ENV.fetch('TOGODX_SERVER_DATABASE_PORT') { 5432 } %>

username: <%= ENV.fetch('TOGODX_SERVER_DATABASE_USER') { 'togodx' } %>

development:

<<: *default

database: togodx_server_development

test:

<<: *default

database: togodx_server_test

production:

<<: *default

database: togodx_server_development

最後の行だけ変更

いっけー！

code:terminal

$ docker-compose -f docker-compose.prod.yml up