Skip to content

Commit

Permalink
Clean dev environment (#3205)
Browse files Browse the repository at this point in the history
* move services to default ports, remove dejavu, remove default hostnames

* Update to postgres 15, add documentation about elasticsearch GUI

* resolve minor documentation suggestions

* move postgres and elasticsearch connect documentation to dev setup, remove unused shell commands from makefile
  • Loading branch information
monneyboi authored Aug 24, 2023
1 parent a43d89c commit eac98ec
Show file tree
Hide file tree
Showing 4 changed files with 22 additions and 36 deletions.
6 changes: 0 additions & 6 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,6 @@ services:
shell: services
$(APPDOCKER) /bin/bash

shell-ui: services
$(UIDOCKER) /bin/bash

shell-db: services
$(COMPOSE) exec postgres psql -U aleph

# To run a single test file:
# make test file=aleph/tests/test_manage.py
test:
Expand Down
17 changes: 3 additions & 14 deletions docker-compose.dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,9 @@ version: "3.2"

services:
postgres:
image: postgres:10.0
hostname: postgres
image: postgres:15
ports:
- "127.0.0.1:15432:5432"
- "127.0.0.1:5432:5432"
volumes:
- postgres-data:/var/lib/postgresql/data
environment:
Expand All @@ -15,7 +14,6 @@ services:

elasticsearch:
image: ghcr.io/alephdata/aleph-elasticsearch:3bb5dbed97cfdb9955324d11e5c623a5c5bbc410
hostname: elasticsearch
restart: on-failure
environment:
- discovery.type=single-node
Expand All @@ -26,18 +24,10 @@ services:
- "http.cors.allow-headers=*"
- "http.cors.allow-credentials=true"
ports:
- "127.0.0.1:19200:9200"
- "127.0.0.1:9200:9200"
volumes:
- elasticsearch-data:/usr/share/elasticsearch/data

dejavu:
image: appbaseio/dejavu:3.6.0
container_name: dejavu
ports:
- '1358:1358'
depends_on:
- elasticsearch

redis:
image: redis:alpine
command: [ "redis-server", "--save", "3600", "10" ]
Expand All @@ -46,7 +36,6 @@ services:

ingest-file:
image: ghcr.io/alephdata/ingest-file:3.19.1
hostname: ingest
tmpfs: /tmp
volumes:
- archive-data:/data
Expand Down
22 changes: 19 additions & 3 deletions docs/src/pages/developers/installation.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -154,13 +154,29 @@ To build the image you can run `make build`, which will build the `alephdata/ale

### Inspecting ElasticSearch

At times it can be useful to inspect the contents of the ElasticSearch indices. The developer setup includes [dejavu](https://github.com/appbaseio/dejavu), a web-based UI for ElasticSearch. In a local environment brought up as described above you should see dejavu under [http://localhost:1358](http://localhost:1358).
At times it can be useful to inspect the contents of the ElasticSearch indices. There are several ElasticSearch GUIs out there that you can use to query ElasticSearch. You could for example use [Dejavu](https://github.com/appbaseio/dejavu):

You will need to pass it the local ES cluster URL (`http://localhost:19200`). You can either inspect an individual index or use `*` to see the whole content. Index names for entity indices follow the pattern `aleph-entity-{{SCHEMA_NAME}}-{{VERSION}}`. For example, the index name for `LegalEntity` entities is `aleph-entity-legalentity-v1`.
```bash
docker run -p 1358:1358 -d appbaseio/dejavu
open http://localhost:1358/
```

Then you can connect Dejavu to the ElasticSearch container that has been exposed on `http://localhost:9200`.

You can also [approach ElasticSearch from the command line](https://www.elastic.co/guide/en/cloud/current/ec-working-with-elasticsearch.html).

### Inspecting the PostgreSQL database

Run `make shell-db` to be dropped into a [psql](https://www.postgresql.org/docs/current/app-psql.html) shell within the database container.
If you want to inspect or manipulate the SQL database directly \(e.g. to edit a user, create or delete a group\), you can connect to the PostgreSQL database.

In development mode, the database is exposed on the host at `127.0.0.1:5432`. \(User, password and database name are all `aleph`\). You can also connect from the shell container:

```bash
make shell
psql $ALEPH_DATABASE_URI
```

The same can be done if you run an instance of the `shell` container in production mode.

## Production deployment

Expand Down
13 changes: 0 additions & 13 deletions docs/src/pages/developers/technical-faq/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -243,19 +243,6 @@ You may also need to run `aleph update` afterwards to refresh some cached inform

That's where it's most at home! We recommend you use the [helm chart](https://github.com/alephdata/aleph/tree/main/helm) to deploy Aleph. It will allow you to override the key settings for your site, while providing a coherent deployment. We use auto-scaling both on the cluster and pod level, which helps to combine fast imports with limited operational cost.

## How can I connect to the database directly?

If you want to manipulate the SQL database directly \(e.g. to edit a user, create or delete a group\), you can connect to the PostgreSQL database.

In development mode, the database is exposed on the host at `127.0.0.1:15432`. \(User, password and database name are all `aleph`\). You can also connect from the shell container:

```bash
make shell
psql $ALEPH_DATABASE_URI
```

The same can be done if you run an instance of the `shell` container in production mode.

## Why do entities have two-part IDs?

When looking at an Aleph URL, you may notice that every entity ID has two parts, separated by a dot \(`.`\), for example:`deadbeef.3cd336a9859bdf2be917f561430f2a83e5da292b`. The first part in this is the actual entity ID, while the second part is a signature \(HMAC\) assigned by the server when indexing the data.
Expand Down

0 comments on commit eac98ec

Please sign in to comment.