Skip to content

S
SMART
Commit f9f8a557 authored by Alexander Lercher's avatar Alexander Lercher
Browse files
Options

Updated deployment scripts

parent 1b4c365e
Hide whitespace changes
Inline Side-by-side
Showing with 58 additions and 89 deletions
README.md
View file @ f9f8a557
......@@ -2,18 +2,18 @@
We propose a framework underlying decentralized social media called SMART, capable of finding relevant interest communities without violating users’ privacy and anonymity. Its objective is to improve trust and eliminate malicious actors in participatory exchanges and collaborative decision making.
## Project structure
The *tools* folder contains supportive scripts used during development and must not be used by any microservice.
### Source code in *src/*
The five folders *data-hub*, *message-broker*, *rest-gateway*, *transaction-hub-in* and *transaction-hub-out* contain the microservices for the five architecture layers respectively.
The *modules* folder contains various Python modules used by multiple microservices to improve code-reuse. When building the Docker images this folder is copied into the app path.
The *tools* folder contains supportive scripts used during development and must not be used by any microservice.
The scripts *build.py* and *deploy.py* are used to create Docker images and deploy these images in Kubernetes respectively.
The *images* folder holds the images used in this document.
### Deployment scripts in *bin/*
The scripts *build.py* and *deploy.py* are used to create Docker images and deploy these images with Kubernetes respectively.
## Overall microservice architecture
![SMART architecture image](images/smart-architecture.png)
![SMART architecture image](documentation/images/smart-architecture.png)
## Trace input handling
1. New traces are POSTed to the REST gateway
......@@ -21,4 +21,4 @@ The *images* folder holds the images used in this document.
1. The trace retrieval microservice receives the message, stores the trace in its document-based database and sends a notification to the message broker
1. The semantic linking microservice receives the notification and GETs all traces (including the new one) from the trace retrieval microservice
1. All traces can now be processed
![Input handling image](images/input-handling.png)
![Input handling image](documentation/images/input-handling.png)
bin/build.py
View file @ f9f8a557
import os
import shutil
docker_reg = 'localhost:5000'
DOCKER_COMPOSE_NAME = "Dockerfile"
PATH = '../src/'
paths = []
for r, _, f in os.walk('./'):
for file in f:
if DOCKER_COMPOSE_NAME == file:
paths.append(os.path.join(r, ''))
for r, _, f in os.walk(PATH):
for filename in f:
if DOCKER_COMPOSE_NAME == filename:
paths.append(os.path.normpath(r))
command_args = [{'path': path,
'name': str(path).split('/')[-2]}
'name': str(path).split(os.path.normpath('/'))[-1]}
for path
in paths]
res_str = []
for command_arg in command_args:
path = command_arg['path']
image_name = f"{docker_reg}/{command_arg['name']}"
image_name = f"{command_arg['name']}"
# copy Dockerfile into root dir to have access to modules folder
shutil.copy2(os.path.join(path, DOCKER_COMPOSE_NAME), './')
os.system(f"docker image build -t {image_name} .")
# build then remove Dockerfile
exit_val = os.system(f"docker image build -t {image_name} .")
os.remove(DOCKER_COMPOSE_NAME)
os.system(f"docker push {image_name}")
res_str .append (f"### {image_name} built with exit code {exit_val}")
print(f"Found {len(command_args)} images")
\ No newline at end of file
# os.system(f"docker push {image_name}")
print(f"Found {len(command_args)} images")
for s in res_str:
print(s)
\ No newline at end of file
src/data-hub/agent-discovery-microservice/Dockerfile
View file @ f9f8a557
......@@ -11,7 +11,7 @@ RUN pip install connexion[swagger-ui]
EXPOSE 5000
WORKDIR /app
COPY data-hub/agent-discovery-microservice/app/ /app/
COPY src/data-hub/agent-discovery-microservice/app/ /app/
RUN chmod a+x main.py
CMD ["python", "./main.py"]
\ No newline at end of file
src/data-hub/agent-discovery-microservice/deployment/deployment.yml
View file @ f9f8a557
......@@ -4,8 +4,6 @@ metadata:
name: agent-discovery
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: agent-discovery
ports:
......@@ -31,6 +29,6 @@ spec:
spec:
containers:
- name: agent-discovery
image: 143.205.173.97:5000/agent-discovery-microservice
image: agent-discovery-microservice
ports:
- containerPort: 5000
\ No newline at end of file
src/data-hub/community-detection-microservice/Dockerfile
View file @ f9f8a557
......@@ -11,7 +11,7 @@ RUN pip install connexion[swagger-ui]
EXPOSE 5000
WORKDIR /app
COPY data-hub/community-detection-microservice/app/ /app/
COPY src/data-hub/community-detection-microservice/app/ /app/
RUN chmod a+x main.py
CMD ["python", "./main.py"]
\ No newline at end of file
src/data-hub/community-detection-microservice/deployment/deployment.yml
View file @ f9f8a557
......@@ -4,8 +4,6 @@ metadata:
name: community-detection
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: community-detection
ports:
......@@ -31,6 +29,6 @@ spec:
spec:
containers:
- name: community-detection
image: 143.205.173.97:5000/community-detection-microservice
image: community-detection-microservice
ports:
- containerPort: 5000
\ No newline at end of file
src/data-hub/democratic-reasoning-microservice/Dockerfile
View file @ f9f8a557
......@@ -11,7 +11,7 @@ RUN pip install connexion[swagger-ui]
EXPOSE 5000
WORKDIR /app
COPY data-hub/democratic-reasoning-microservice/app/ /app/
COPY src/data-hub/democratic-reasoning-microservice/app/ /app/
RUN chmod a+x main.py
CMD ["python", "./main.py"]
\ No newline at end of file
src/data-hub/democratic-reasoning-microservice/deployment/deployment.yml
View file @ f9f8a557
......@@ -4,8 +4,6 @@ metadata:
name: democratic-reasoning
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: democratic-reasoning
ports:
......@@ -31,6 +29,6 @@ spec:
spec:
containers:
- name: democratic-reasoning
image: 143.205.173.97:5000/democratic-reasoning-microservice
image: democratic-reasoning-microservice
ports:
- containerPort: 5000
\ No newline at end of file
src/data-hub/geo-profiling-microservice/Dockerfile
View file @ f9f8a557
......@@ -11,7 +11,7 @@ RUN pip install connexion[swagger-ui]
EXPOSE 5000
WORKDIR /app
COPY data-hub/geo-profiling-microservice/app/ /app/
COPY src/data-hub/geo-profiling-microservice/app/ /app/
RUN chmod a+x main.py
CMD ["python", "./main.py"]
\ No newline at end of file
src/data-hub/geo-profiling-microservice/deployment/deployment.yml
View file @ f9f8a557
......@@ -4,8 +4,6 @@ metadata:
name: geo-profiling
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: geo-profiling
ports:
......@@ -31,6 +29,6 @@ spec:
spec:
containers:
- name: geo-profiling
image: 143.205.173.97:5000/geo-profiling-microservice
image: geo-profiling-microservice
ports:
- containerPort: 5000
\ No newline at end of file
src/data-hub/pareto-trust-microservice/Dockerfile
View file @ f9f8a557
......@@ -11,7 +11,7 @@ RUN pip install connexion[swagger-ui]
EXPOSE 5000
WORKDIR /app
COPY data-hub/pareto-trust-microservice/app/ /app/
COPY src/data-hub/pareto-trust-microservice/app/ /app/
RUN chmod a+x main.py
CMD ["python", "./main.py"]
\ No newline at end of file
src/data-hub/pareto-trust-microservice/deployment/deployment.yml
View file @ f9f8a557
......@@ -4,8 +4,6 @@ metadata:
name: pareto-trust
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: pareto-trust
ports:
......@@ -31,6 +29,6 @@ spec:
spec:
containers:
- name: pareto-trust
image: 143.205.173.97:5000/pareto-trust-microservice
image: pareto-trust-microservice
ports:
- containerPort: 5000
\ No newline at end of file
src/data-hub/reputation-calculation-microservice/Dockerfile
View file @ f9f8a557
......@@ -11,7 +11,7 @@ RUN pip install connexion[swagger-ui]
EXPOSE 5000
WORKDIR /app
COPY data-hub/reputation-calculation-microservice/app/ /app/
COPY src/data-hub/reputation-calculation-microservice/app/ /app/
RUN chmod a+x main.py
CMD ["python", "./main.py"]
\ No newline at end of file
src/data-hub/reputation-calculation-microservice/deployment/deployment.yml
View file @ f9f8a557
......@@ -4,8 +4,6 @@ metadata:
name: reputation-calculation
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: reputation-calculation
ports:
......@@ -31,6 +29,6 @@ spec:
spec:
containers:
- name: reputation-calculation
image: 143.205.173.97:5000/reputation-calculation-microservice
image: reputation-calculation-microservice
ports:
- containerPort: 5000
\ No newline at end of file
src/data-hub/role-discovery-microservice/Dockerfile
View file @ f9f8a557
......@@ -11,7 +11,7 @@ RUN pip install connexion[swagger-ui]
EXPOSE 5000
WORKDIR /app
COPY data-hub/role-discovery-microservice/app/ /app/
COPY src/data-hub/role-discovery-microservice/app/ /app/
RUN chmod a+x main.py
CMD ["python", "./main.py"]
\ No newline at end of file
src/data-hub/role-discovery-microservice/deployment/deployment.yml
View file @ f9f8a557
......@@ -4,8 +4,6 @@ metadata:
name: role-discovery
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: role-discovery
ports:
......@@ -31,6 +29,6 @@ spec:
spec:
containers:
- name: role-discovery
image: 143.205.173.97:5000/role-discovery-microservice
image: role-discovery-microservice
ports:
- containerPort: 5000
\ No newline at end of file
src/data-hub/semantic-linking-microservice/Dockerfile
View file @ f9f8a557
......@@ -12,7 +12,7 @@ RUN pip install pika
EXPOSE 5000
WORKDIR /app
COPY data-hub/semantic-linking-microservice/app/ /app/
COPY src/data-hub/semantic-linking-microservice/app/ /app/
COPY modules/ /app/
RUN chmod a+x main.py
......
src/data-hub/semantic-linking-microservice/deployment/deployment.yml
View file @ f9f8a557
......@@ -4,8 +4,6 @@ metadata:
name: semantic-linking
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: semantic-linking
ports:
......@@ -31,6 +29,6 @@ spec:
spec:
containers:
- name: semantic-linking
image: 143.205.173.97:5000/semantic-linking-microservice
image: semantic-linking-microservice
ports:
- containerPort: 5000
\ No newline at end of file
src/data-hub/stage-discovery-microservice/Dockerfile
View file @ f9f8a557
......@@ -11,7 +11,7 @@ RUN pip install connexion[swagger-ui]
EXPOSE 5000
WORKDIR /app
COPY data-hub/stage-discovery-microservice/app/ /app/
COPY src/data-hub/stage-discovery-microservice/app/ /app/
RUN chmod a+x main.py
CMD ["python", "./main.py"]
\ No newline at end of file
src/data-hub/stage-discovery-microservice/deployment/deployment.yml
View file @ f9f8a557
......@@ -4,8 +4,6 @@ metadata:
name: stage-discovery
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: stage-discovery
ports:
......@@ -31,6 +29,6 @@ spec:
spec:
containers:
- name: stage-discovery
image: 143.205.173.97:5000/stage-discovery-microservice
image: stage-discovery-microservice
ports:
- containerPort: 5000
\ No newline at end of file
src/message-broker/deployment/deployment.yml
View file @ f9f8a557
......@@ -4,8 +4,6 @@ metadata:
name: rabbit-mq
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: rabbit-mq
ports:
......
src/rest-gateway/Dockerfile
View file @ f9f8a557
......@@ -13,7 +13,7 @@ RUN pip install deprecated
EXPOSE 5000
WORKDIR /app
COPY rest-gateway/app/ /app/
COPY src/rest-gateway/app/ /app/
COPY modules/ /app/
RUN chmod a+x main.py
......
src/rest-gateway/deployment/deployment.yml
View file @ f9f8a557
......@@ -4,8 +4,6 @@ metadata:
name: rest-gateway
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: rest-gateway
ports:
......@@ -31,6 +29,6 @@ spec:
spec:
containers:
- name: rest-gateway
image: 143.205.173.97:5000/rest-gateway
image: rest-gateway
ports:
- containerPort: 5000
\ No newline at end of file
src/transaction-hub-in/network-metrics-retrieval-microservice/Dockerfile
View file @ f9f8a557
......@@ -11,7 +11,7 @@ RUN pip install connexion[swagger-ui]
EXPOSE 5000
WORKDIR /app
COPY transaction-hub-in/network-metrics-retrieval-microservice/app/ /app/
COPY src/transaction-hub-in/network-metrics-retrieval-microservice/app/ /app/
RUN chmod a+x main.py
CMD ["python", "./main.py"]
\ No newline at end of file
src/transaction-hub-in/network-metrics-retrieval-microservice/deployment/deployment.yml
View file @ f9f8a557
......@@ -4,8 +4,6 @@ metadata:
name: network-metrics-retrieval
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: network-metrics-retrieval
ports:
......@@ -31,6 +29,6 @@ spec:
spec:
containers:
- name: network-metrics-retrieval
image: 143.205.173.97:5000/network-metrics-retrieval-microservice
image: network-metrics-retrieval-microservice
ports:
- containerPort: 5000
\ No newline at end of file
src/transaction-hub-in/trace-retrieval-microservice/Dockerfile
View file @ f9f8a557
......@@ -14,7 +14,7 @@ RUN pip install deprecated
EXPOSE 5000
WORKDIR /app
COPY transaction-hub-in/trace-retrieval-microservice/app/ /app/
COPY src/transaction-hub-in/trace-retrieval-microservice/app/ /app/
COPY modules/ /app/
RUN chmod a+x main.py
......
src/transaction-hub-in/trace-retrieval-microservice/deployment/deployment.yml
View file @ f9f8a557
......@@ -4,8 +4,6 @@ metadata:
name: trace-retrieval
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: trace-retrieval
ports:
......@@ -31,7 +29,7 @@ spec:
spec:
containers:
- name: trace-retrieval
image: 143.205.173.97:5000/trace-retrieval-microservice
image: trace-retrieval-microservice
ports:
- containerPort: 5000
imagePullPolicy: Always
......@@ -42,8 +40,6 @@ metadata:
name: trace-retrieval-db
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: trace-retrieval-db
ports:
......
src/transaction-hub-out/data-access-microservice/Dockerfile
View file @ f9f8a557
......@@ -11,7 +11,7 @@ RUN pip install connexion[swagger-ui]
EXPOSE 5000
WORKDIR /app
COPY transaction-hub-out/data-access-microservice/app/ /app/
COPY src/transaction-hub-out/data-access-microservice/app/ /app/
RUN chmod a+x main.py
CMD ["python", "./main.py"]
\ No newline at end of file
src/transaction-hub-out/data-access-microservice/deployment/deployment.yml
View file @ f9f8a557
......@@ -4,8 +4,6 @@ metadata:
name: data-access
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: data-access
ports:
......@@ -31,6 +29,6 @@ spec:
spec:
containers:
- name: data-access
image: 143.205.173.97:5000/data-access-microservice
image: data-access-microservice
ports:
- containerPort: 5000
\ No newline at end of file
src/transaction-hub-out/event-detection-microservice/Dockerfile
View file @ f9f8a557
......@@ -11,7 +11,7 @@ RUN pip install connexion[swagger-ui]
EXPOSE 5000
WORKDIR /app
COPY transaction-hub-out/event-detection-microservice/app/ /app/
COPY src/transaction-hub-out/event-detection-microservice/app/ /app/
RUN chmod a+x main.py
CMD ["python", "./main.py"]
\ No newline at end of file
src/transaction-hub-out/event-detection-microservice/deployment/deployment.yml
View file @ f9f8a557
......@@ -4,8 +4,6 @@ metadata:
name: event-detection
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: event-detection
ports:
......@@ -31,6 +29,6 @@ spec:
spec:
containers:
- name: event-detection
image: 143.205.173.97:5000/event-detection-microservice
image: event-detection-microservice
ports:
- containerPort: 5000
\ No newline at end of file
src/transaction-hub-out/smart-matching-microservice/Dockerfile
View file @ f9f8a557
......@@ -11,7 +11,7 @@ RUN pip install connexion[swagger-ui]
EXPOSE 5000
WORKDIR /app
COPY transaction-hub-out/smart-matching-microservice/app/ /app/
COPY src/transaction-hub-out/smart-matching-microservice/app/ /app/
RUN chmod a+x main.py
CMD ["python", "./main.py"]
\ No newline at end of file
src/transaction-hub-out/smart-matching-microservice/deployment/deployment.yml
View file @ f9f8a557
......@@ -4,8 +4,6 @@ metadata:
name: smart-matching
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: smart-matching
ports:
......@@ -31,6 +29,6 @@ spec:
spec:
containers:
- name: smart-matching
image: 143.205.173.97:5000/smart-matching-microservice
image: smart-matching-microservice
ports:
- containerPort: 5000
\ No newline at end of file
src/transaction-hub-out/tokenized-decision-making-microservice/Dockerfile
View file @ f9f8a557
......@@ -11,7 +11,7 @@ RUN pip install connexion[swagger-ui]
EXPOSE 5000
WORKDIR /app
COPY transaction-hub-out/tokenized-decision-making-microservice/app/ /app/
COPY src/transaction-hub-out/tokenized-decision-making-microservice/app/ /app/
RUN chmod a+x main.py
CMD ["python", "./main.py"]
\ No newline at end of file
src/transaction-hub-out/tokenized-decision-making-microservice/deployment/deployment.yml
View file @ f9f8a557
......@@ -4,8 +4,6 @@ metadata:
name: tokenized-decision-making
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: tokenized-decision-making
ports:
......@@ -31,6 +29,6 @@ spec:
spec:
containers:
- name: tokenized-decision-making
image: 143.205.173.97:5000/tokenized-decision-making-microservice
image: tokenized-decision-making-microservice
ports:
- containerPort: 5000
\ No newline at end of file
src/transaction-hub-out/trust-sla-microservice/Dockerfile
View file @ f9f8a557
......@@ -11,7 +11,7 @@ RUN pip install connexion[swagger-ui]
EXPOSE 5000
WORKDIR /app
COPY transaction-hub-out/trust-sla-microservice/app/ /app/
COPY src/transaction-hub-out/trust-sla-microservice/app/ /app/
RUN chmod a+x main.py
CMD ["python", "./main.py"]
\ No newline at end of file
src/transaction-hub-out/trust-sla-microservice/deployment/deployment.yml
View file @ f9f8a557
......@@ -4,8 +4,6 @@ metadata:
name: trust-sla
spec:
type: LoadBalancer
externalIPs:
- 143.205.173.102
selector:
app: trust-sla
ports:
......@@ -31,6 +29,6 @@ spec:
spec:
containers:
- name: trust-sla
image: 143.205.173.97:5000/trust-sla-microservice
image: trust-sla-microservice
ports:
- containerPort: 5000
\ No newline at end of file
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment