Commit f9f8a557 authored by Alexander Lercher's avatar Alexander Lercher

Updated deployment scripts

parent 1b4c365e
......@@ -2,18 +2,18 @@
We propose a framework underlying decentralized social media called SMART, capable of finding relevant interest communities without violating users’ privacy and anonymity. Its objective is to improve trust and eliminate malicious actors in participatory exchanges and collaborative decision making.
## Project structure
The *tools* folder contains supportive scripts used during development and must not be used by any microservice.
### Source code in *src/*
The five folders *data-hub*, *message-broker*, *rest-gateway*, *transaction-hub-in* and *transaction-hub-out* contain the microservices for the five architecture layers respectively.
The *modules* folder contains various Python modules used by multiple microservices to improve code-reuse. When building the Docker images this folder is copied into the app path.
The *tools* folder contains supportive scripts used during development and must not be used by any microservice.
The scripts *build.py* and *deploy.py* are used to create Docker images and deploy these images in Kubernetes respectively.
The *images* folder holds the images used in this document.
### Deployment scripts in *bin/*
The scripts *build.py* and *deploy.py* are used to create Docker images and deploy these images with Kubernetes respectively.
## Overall microservice architecture
![SMART architecture image](images/smart-architecture.png)
![SMART architecture image](documentation/images/smart-architecture.png)
## Trace input handling
1. New traces are POSTed to the REST gateway
......@@ -21,4 +21,4 @@ The *images* folder holds the images used in this document.
1. The trace retrieval microservice receives the message, stores the trace in its document-based database and sends a notification to the message broker
1. The semantic linking microservice receives the notification and GETs all traces (including the new one) from the trace retrieval microservice
1. All traces can now be processed
![Input handling image](images/input-handling.png)
![Input handling image](documentation/images/input-handling.png)
import os
import shutil
docker_reg = 'localhost:5000'
DOCKER_COMPOSE_NAME = "Dockerfile"
PATH = '../src/'
paths = []
for r, _, f in os.walk('./'):
for file in f:
if DOCKER_COMPOSE_NAME == file:
paths.append(os.path.join(r, ''))
for r, _, f in os.walk(PATH):
for filename in f:
if DOCKER_COMPOSE_NAME == filename:
paths.append(os.path.normpath(r))
command_args = [{'path': path,
'name': str(path).split('/')[-2]}
'name': str(path).split(os.path.normpath('/'))[-1]}
for path
in paths]
res_str = []
for command_arg in command_args:
path = command_arg['path']
image_name = f"{docker_reg}/{command_arg['name']}"
image_name = f"{command_arg['name']}"
# copy Dockerfile into root dir to have access to modules folder
shutil.copy2(os.path.join(path, DOCKER_COMPOSE_NAME), './')
os.system(f"docker image build -t {image_name} .")
# build then remove Dockerfile
exit_val = os.system(f"docker image build -t {image_name} .")
os.remove(DOCKER_COMPOSE_NAME)
os.system(f"docker push {image_name}")
res_str .append (f"### {image_name} built with exit code {exit_val}")
# os.system(f"docker push {image_name}")
print(f"Found {len(command_args)} images")
for s in res_str:
print(s)
\ No newline at end of file
......@@ -11,7 +11,7 @@ RUN pip install connexion[swagger-ui]
EXPOSE 5000
WORKDIR /app
COPY data-hub/agent-discovery-microservice/app/ /app/
COPY src/data-hub/agent-discovery-microservice/app/ /app/
RUN chmod a+x main.py
CMD ["python", "./main.py"]
\ No newline at end of file
......@@ -4,8 +4,6 @@ metadata:
name: agent-discovery
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: agent-discovery
ports:
......@@ -31,6 +29,6 @@ spec:
spec:
containers:
- name: agent-discovery
image: 143.205.173.97:5000/agent-discovery-microservice
image: agent-discovery-microservice
ports:
- containerPort: 5000
\ No newline at end of file
......@@ -11,7 +11,7 @@ RUN pip install connexion[swagger-ui]
EXPOSE 5000
WORKDIR /app
COPY data-hub/community-detection-microservice/app/ /app/
COPY src/data-hub/community-detection-microservice/app/ /app/
RUN chmod a+x main.py
CMD ["python", "./main.py"]
\ No newline at end of file
......@@ -4,8 +4,6 @@ metadata:
name: community-detection
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: community-detection
ports:
......@@ -31,6 +29,6 @@ spec:
spec:
containers:
- name: community-detection
image: 143.205.173.97:5000/community-detection-microservice
image: community-detection-microservice
ports:
- containerPort: 5000
\ No newline at end of file
......@@ -11,7 +11,7 @@ RUN pip install connexion[swagger-ui]
EXPOSE 5000
WORKDIR /app
COPY data-hub/democratic-reasoning-microservice/app/ /app/
COPY src/data-hub/democratic-reasoning-microservice/app/ /app/
RUN chmod a+x main.py
CMD ["python", "./main.py"]
\ No newline at end of file
......@@ -4,8 +4,6 @@ metadata:
name: democratic-reasoning
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: democratic-reasoning
ports:
......@@ -31,6 +29,6 @@ spec:
spec:
containers:
- name: democratic-reasoning
image: 143.205.173.97:5000/democratic-reasoning-microservice
image: democratic-reasoning-microservice
ports:
- containerPort: 5000
\ No newline at end of file
......@@ -11,7 +11,7 @@ RUN pip install connexion[swagger-ui]
EXPOSE 5000
WORKDIR /app
COPY data-hub/geo-profiling-microservice/app/ /app/
COPY src/data-hub/geo-profiling-microservice/app/ /app/
RUN chmod a+x main.py
CMD ["python", "./main.py"]
\ No newline at end of file
......@@ -4,8 +4,6 @@ metadata:
name: geo-profiling
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: geo-profiling
ports:
......@@ -31,6 +29,6 @@ spec:
spec:
containers:
- name: geo-profiling
image: 143.205.173.97:5000/geo-profiling-microservice
image: geo-profiling-microservice
ports:
- containerPort: 5000
\ No newline at end of file
......@@ -11,7 +11,7 @@ RUN pip install connexion[swagger-ui]
EXPOSE 5000
WORKDIR /app
COPY data-hub/pareto-trust-microservice/app/ /app/
COPY src/data-hub/pareto-trust-microservice/app/ /app/
RUN chmod a+x main.py
CMD ["python", "./main.py"]
\ No newline at end of file
......@@ -4,8 +4,6 @@ metadata:
name: pareto-trust
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: pareto-trust
ports:
......@@ -31,6 +29,6 @@ spec:
spec:
containers:
- name: pareto-trust
image: 143.205.173.97:5000/pareto-trust-microservice
image: pareto-trust-microservice
ports:
- containerPort: 5000
\ No newline at end of file
......@@ -11,7 +11,7 @@ RUN pip install connexion[swagger-ui]
EXPOSE 5000
WORKDIR /app
COPY data-hub/reputation-calculation-microservice/app/ /app/
COPY src/data-hub/reputation-calculation-microservice/app/ /app/
RUN chmod a+x main.py
CMD ["python", "./main.py"]
\ No newline at end of file
......@@ -4,8 +4,6 @@ metadata:
name: reputation-calculation
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: reputation-calculation
ports:
......@@ -31,6 +29,6 @@ spec:
spec:
containers:
- name: reputation-calculation
image: 143.205.173.97:5000/reputation-calculation-microservice
image: reputation-calculation-microservice
ports:
- containerPort: 5000
\ No newline at end of file
......@@ -11,7 +11,7 @@ RUN pip install connexion[swagger-ui]
EXPOSE 5000
WORKDIR /app
COPY data-hub/role-discovery-microservice/app/ /app/
COPY src/data-hub/role-discovery-microservice/app/ /app/
RUN chmod a+x main.py
CMD ["python", "./main.py"]
\ No newline at end of file
......@@ -4,8 +4,6 @@ metadata:
name: role-discovery
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: role-discovery
ports:
......@@ -31,6 +29,6 @@ spec:
spec:
containers:
- name: role-discovery
image: 143.205.173.97:5000/role-discovery-microservice
image: role-discovery-microservice
ports:
- containerPort: 5000
\ No newline at end of file
......@@ -12,7 +12,7 @@ RUN pip install pika
EXPOSE 5000
WORKDIR /app
COPY data-hub/semantic-linking-microservice/app/ /app/
COPY src/data-hub/semantic-linking-microservice/app/ /app/
COPY modules/ /app/
RUN chmod a+x main.py
......
......@@ -4,8 +4,6 @@ metadata:
name: semantic-linking
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: semantic-linking
ports:
......@@ -31,6 +29,6 @@ spec:
spec:
containers:
- name: semantic-linking
image: 143.205.173.97:5000/semantic-linking-microservice
image: semantic-linking-microservice
ports:
- containerPort: 5000
\ No newline at end of file
......@@ -11,7 +11,7 @@ RUN pip install connexion[swagger-ui]
EXPOSE 5000
WORKDIR /app
COPY data-hub/stage-discovery-microservice/app/ /app/
COPY src/data-hub/stage-discovery-microservice/app/ /app/
RUN chmod a+x main.py
CMD ["python", "./main.py"]
\ No newline at end of file
......@@ -4,8 +4,6 @@ metadata:
name: stage-discovery
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: stage-discovery
ports:
......@@ -31,6 +29,6 @@ spec:
spec:
containers:
- name: stage-discovery
image: 143.205.173.97:5000/stage-discovery-microservice
image: stage-discovery-microservice
ports:
- containerPort: 5000
\ No newline at end of file
......@@ -4,8 +4,6 @@ metadata:
name: rabbit-mq
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: rabbit-mq
ports:
......
......@@ -13,7 +13,7 @@ RUN pip install deprecated
EXPOSE 5000
WORKDIR /app
COPY rest-gateway/app/ /app/
COPY src/rest-gateway/app/ /app/
COPY modules/ /app/
RUN chmod a+x main.py
......
......@@ -4,8 +4,6 @@ metadata:
name: rest-gateway
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: rest-gateway
ports:
......@@ -31,6 +29,6 @@ spec:
spec:
containers:
- name: rest-gateway
image: 143.205.173.97:5000/rest-gateway
image: rest-gateway
ports:
- containerPort: 5000
\ No newline at end of file
......@@ -11,7 +11,7 @@ RUN pip install connexion[swagger-ui]
EXPOSE 5000
WORKDIR /app
COPY transaction-hub-in/network-metrics-retrieval-microservice/app/ /app/
COPY src/transaction-hub-in/network-metrics-retrieval-microservice/app/ /app/
RUN chmod a+x main.py
CMD ["python", "./main.py"]
\ No newline at end of file
......@@ -4,8 +4,6 @@ metadata:
name: network-metrics-retrieval
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: network-metrics-retrieval
ports:
......@@ -31,6 +29,6 @@ spec:
spec:
containers:
- name: network-metrics-retrieval
image: 143.205.173.97:5000/network-metrics-retrieval-microservice
image: network-metrics-retrieval-microservice
ports:
- containerPort: 5000
\ No newline at end of file
......@@ -14,7 +14,7 @@ RUN pip install deprecated
EXPOSE 5000
WORKDIR /app
COPY transaction-hub-in/trace-retrieval-microservice/app/ /app/
COPY src/transaction-hub-in/trace-retrieval-microservice/app/ /app/
COPY modules/ /app/
RUN chmod a+x main.py
......
......@@ -4,8 +4,6 @@ metadata:
name: trace-retrieval
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: trace-retrieval
ports:
......@@ -31,7 +29,7 @@ spec:
spec:
containers:
- name: trace-retrieval
image: 143.205.173.97:5000/trace-retrieval-microservice
image: trace-retrieval-microservice
ports:
- containerPort: 5000
imagePullPolicy: Always
......@@ -42,8 +40,6 @@ metadata:
name: trace-retrieval-db
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: trace-retrieval-db
ports:
......
......@@ -11,7 +11,7 @@ RUN pip install connexion[swagger-ui]
EXPOSE 5000
WORKDIR /app
COPY transaction-hub-out/data-access-microservice/app/ /app/
COPY src/transaction-hub-out/data-access-microservice/app/ /app/
RUN chmod a+x main.py
CMD ["python", "./main.py"]
\ No newline at end of file
......@@ -4,8 +4,6 @@ metadata:
name: data-access
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: data-access
ports:
......@@ -31,6 +29,6 @@ spec:
spec:
containers:
- name: data-access
image: 143.205.173.97:5000/data-access-microservice
image: data-access-microservice
ports:
- containerPort: 5000
\ No newline at end of file
......@@ -11,7 +11,7 @@ RUN pip install connexion[swagger-ui]
EXPOSE 5000
WORKDIR /app
COPY transaction-hub-out/event-detection-microservice/app/ /app/
COPY src/transaction-hub-out/event-detection-microservice/app/ /app/
RUN chmod a+x main.py
CMD ["python", "./main.py"]
\ No newline at end of file
......@@ -4,8 +4,6 @@ metadata:
name: event-detection
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: event-detection
ports:
......@@ -31,6 +29,6 @@ spec:
spec:
containers:
- name: event-detection
image: 143.205.173.97:5000/event-detection-microservice
image: event-detection-microservice
ports:
- containerPort: 5000
\ No newline at end of file
......@@ -11,7 +11,7 @@ RUN pip install connexion[swagger-ui]
EXPOSE 5000
WORKDIR /app
COPY transaction-hub-out/smart-matching-microservice/app/ /app/
COPY src/transaction-hub-out/smart-matching-microservice/app/ /app/
RUN chmod a+x main.py
CMD ["python", "./main.py"]
\ No newline at end of file
......@@ -4,8 +4,6 @@ metadata:
name: smart-matching
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: smart-matching
ports:
......@@ -31,6 +29,6 @@ spec:
spec:
containers:
- name: smart-matching
image: 143.205.173.97:5000/smart-matching-microservice
image: smart-matching-microservice
ports:
- containerPort: 5000
\ No newline at end of file
......@@ -11,7 +11,7 @@ RUN pip install connexion[swagger-ui]
EXPOSE 5000
WORKDIR /app
COPY transaction-hub-out/tokenized-decision-making-microservice/app/ /app/
COPY src/transaction-hub-out/tokenized-decision-making-microservice/app/ /app/
RUN chmod a+x main.py
CMD ["python", "./main.py"]
\ No newline at end of file
......@@ -4,8 +4,6 @@ metadata:
name: tokenized-decision-making
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: tokenized-decision-making
ports:
......@@ -31,6 +29,6 @@ spec:
spec:
containers:
- name: tokenized-decision-making
image: 143.205.173.97:5000/tokenized-decision-making-microservice
image: tokenized-decision-making-microservice
ports:
- containerPort: 5000
\ No newline at end of file
......@@ -11,7 +11,7 @@ RUN pip install connexion[swagger-ui]
EXPOSE 5000
WORKDIR /app
COPY transaction-hub-out/trust-sla-microservice/app/ /app/
COPY src/transaction-hub-out/trust-sla-microservice/app/ /app/
RUN chmod a+x main.py
CMD ["python", "./main.py"]
\ No newline at end of file
......@@ -4,8 +4,6 @@ metadata:
name: trust-sla
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: trust-sla
ports:
......@@ -31,6 +29,6 @@ spec:
spec:
containers:
- name: trust-sla
image: 143.205.173.97:5000/trust-sla-microservice
image: trust-sla-microservice
ports:
- containerPort: 5000
\ No newline at end of file
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment