QUESTION: Graceful way to check for several things on startup

[bkraul]

@kevinpapst I was trying to find a question template for this, as this is not a bug. I am currently working on an nginx-based docker image. However, I want to make the deployment as hassle free as possible, controlled by the docker ENV vars.

I am using a base image that allows me to specify a number of entry point scripts (to be executed on launch of the container). Everything works, but after the first time, as the script tries to create schema, migrations, admin user (specified in docker-compose) it is obviously going to fail. The container works just fine, though.

However, the schema:create command clearly specifies "do not run this on a production environment", I would assume because it might break it?

Therefore, I am looking for a graceful way to check for:

• Database exists.
• Update schema exists.
• Migration is up to date.
• First superadmin (local) user exists.

This, using preferable a simple bash script, but I am open to a php script as well.

I think I have also found a way to send the output of the current log (env/prod) to the stdout. The base image I have uses supervisor. If I put the tail of of the log into a supervisor job, it will live for the duration of the container, and will display wherever the container's stdout is shown.

[kevinpapst]

I was working on an installation command that does exactly what you said (except the user part).
Please test if you can work with bin/console kimai:install.
There is no way (that I know of) to achieve this without code'ing.

[kevinpapst]

It might need adjustment. And it has a part to check permissions, which is likely not working.
Thats why I announced it nowhere, because its not completely ready yet.
But maybe we can get it together to a point where it is safely usable.

[bkraul]

Thanks! I will give the command a try. It could be good to add the option to create the first super admin during the install (that way its existence would be properly checked as well). Looking at the source of InstallCommand.php (not very familiar with Symfony).

[kevinpapst]

The first user that is created through the WebUI gets super_admin permissions automatically, thats why I skipped it for now

[bkraul]

Correct but in the case of an automation (where the deployment requires that no registration is enabled from the getgo) it would be helpful, but I understand not necessarily critical.

[bkraul]

Trying to use the kimai:install method but I get this in my container's stdout:

web_1  | Welcome to the interactive Kimai installer!
web_1  | ===========================================
web_1  |
web_1  |  [ERROR] Installation only works in interactive mode

[kevinpapst]

Can you test the changes from #932 ?

[bkraul]

Hey man, sorry I missed this. I am not sure exactly what I was supposed to test on #932, since I am using an LDAP setup and the issue involves non-LDAP environments. I see you merged it tho. I have already rebuilt the images and pushed them to my docker repo.

[kevinpapst]

Nope, check again #932 - thats about the installer command.
I accidentaly linked #931 before ... that was something different.

[bkraul]

Aaah! will check that and get with you. Thanks!

[bkraul]

Tested it. Works for both new installs (empty database pre-created in sql client), and existing database. The only thing I have to do is continue to attempt to create the admin user (get an error if existing but there is no harm).

You da man! Awaiting merge.

[tobybatch]

@kevinpapst I see that kimai:install is non destructive. It looks like it runs migration if they are needed is that right?

Can I run it against existing DBs during start up to check (and run) migrations for me if needed?

[bkraul]

That's what I am doing on the nginx/apache ones. Doesn't seem to affect existing database setups. It instead says it's found existing database/tables, etc, and says it is skipping. But I guess it would be good to get a confirmation (I mean from @kevinpapst, not in the script, hehe, I love it supports non-interactive now).

@kevinpapst thanks for merging into master. It makes things way easier in setting up turn-key.

[kevinpapst]

Nope, unfortunately not. As I didn't knew the --allow-no-migration flag before you posted that in the docker repo, it doesn't do that by now.

This method needs to be adjusted:
https://github.com/kevinpapst/kimai2/blob/master/src/Command/InstallCommand.php#L225

Change the logic:
"add all" if the table is empty, otherwise run "migrate --allow-no-migration"

Wanna send a PR? If not, please open a new feature request.

[kevinpapst]

Forget the request, I added a PR. Can you test #944 ?

[tobybatch]

Will do, I'm in the middle of a massive refactor on the dockers atm. I'll do that later or tomorrow.

[kevinpapst]

You can test master tomorrow, I needed the change and merged it already

[bkraul]

So this is in master, right? I can just rebuild images to test?

[kevinpapst]

Exactly

[bkraul]

Everything seems to work OK. Output of my kimai docker container stdout:

Kimai installer - v1.0.1
========================
 ! [NOTE] Database is existing and connection could be established              
 ! [NOTE] It seems as if you already have the required tables in your database, 
 !        skipping schema creation                                              
                    Application Migrations                    

Migrating up to 20190706224219 from 20190617100845

  ++ migrating 20190706224211 (Fix meta-table definitions)
     -> ALTER TABLE kimai2_activities_meta CHANGE activity_id activity_id INT NOT NULL, CHANGE visible visible TINYINT(1) DEFAULT '0' NOT NULL
     -> ALTER TABLE kimai2_customers_meta CHANGE customer_id customer_id INT NOT NULL, CHANGE visible visible TINYINT(1) DEFAULT '0' NOT NULL
     -> ALTER TABLE kimai2_projects_meta CHANGE project_id project_id INT NOT NULL, CHANGE visible visible TINYINT(1) DEFAULT '0' NOT NULL
     -> ALTER TABLE kimai2_timesheet_meta CHANGE timesheet_id timesheet_id INT NOT NULL, CHANGE visible visible TINYINT(1) DEFAULT '0' NOT NULL

  ++ migrated (took 708.4ms, used 40.25M memory)
  ++ migrating 20190706224219 (Creates several indices to improve speed for default queries.)

     -> CREATE INDEX IDX_8811FE1C7AB0E859166D1F9C ON kimai2_activities (visible, project_id)
     -> CREATE INDEX IDX_8811FE1C7AB0E859166D1F9C5E237E06 ON kimai2_activities (visible, project_id, name)
     -> CREATE INDEX IDX_8811FE1C7AB0E8595E237E06 ON kimai2_activities (visible, name)
     -> CREATE INDEX IDX_5A9760447AB0E859 ON kimai2_customers (visible)
     -> CREATE INDEX IDX_407F12069395C3F37AB0E8595E237E06 ON kimai2_projects (customer_id, visible, name)
     -> CREATE INDEX IDX_407F12069395C3F37AB0E859BF396750 ON kimai2_projects (customer_id, visible, id)
     -> CREATE INDEX IDX_4F60C6B18D93D649502DF587 ON kimai2_timesheet (user, start_time)
     -> CREATE INDEX IDX_4F60C6B1502DF587 ON kimai2_timesheet (start_time)
     -> CREATE INDEX IDX_4F60C6B1502DF58741561401 ON kimai2_timesheet (start_time, end_time)
     -> CREATE INDEX IDX_4F60C6B1502DF587415614018D93D649 ON kimai2_timesheet (start_time, end_time, user)

  ++ migrated (took 1659.4ms, used 42.25M memory)
  ------------------------
  ++ finished in 1662ms
  ++ used 42.25M memory
  ++ 2 migrations executed
  ++ 14 sql queries

 Rebuilding your cache now, please be patient ...

 // Clearing the cache for the prod environment with debug                      
 // false                                                                       

 [OK] Cache for the "prod" environment (debug=false) was successfully cleared.  

 // Warming up the cache for the prod environment with debug                    
 // false                                                                       

 [OK] Cache for the "prod" environment (debug=false) was successfully warmed.   
 [OK] Congratulations! Kimai 2 (1.0.1 stable) was successful installed!         

[kevinpapst]

did you try to re-start once more, when there is no more migration available?

[bkraul]

Yes:

Kimai installer - v1.0.1
========================

 ! [NOTE] Database is existing and connection could be established              
 ! [NOTE] It seems as if you already have the required tables in your database, 
 !        skipping schema creation                                              
                    Application Migrations                    
No migrations to execute.
 Rebuilding your cache now, please be patient ...

 // Clearing the cache for the prod environment with debug                      
 // false                                                                       

 [OK] Cache for the "prod" environment (debug=false) was successfully cleared.  

 // Warming up the cache for the prod environment with debug                    
 // false                                                                       

 [OK] Cache for the "prod" environment (debug=false) was successfully warmed.   

 [OK] Congratulations! Kimai 2 (1.0.1 stable) was successful installed!         

[kevinpapst]

Awesome, thanks 👍

[bkraul]

Thank you, man! This a pretty cool project!

[kevinpapst]

It just gets better with more input and ideas from the community

[bkraul]

@kevinpapst I have finally implemented kimai2 log (whether dev or prod) to the container's stdout. You can check out the Dockefile/scripts here, for anyone who wants to do the same thing:

https://git.belmankraul.com/docker/kimai2

[kevinpapst]

Nice!

If that is a general purpose Kimai container, you can send a PR to:
https://github.com/kimai/www.kimai.org/blob/master/_documentation/docker.md
adding your solution.

[bkraul]

Since it is not based on a vanilla nginx/apache php image, I am not sure if you'd consider it general purpose, unless what you mean is that it is generic, which that it is.

It's based on the webdevops images. I am not using php:7.2.9-apache-stretch as base, like @tobybatch. My implementation requires the use of supervisor to control the multiple services running (apache/nginx, php-fpm, the kimai2 log, etc).

But, if y'all are interested in me making it available on github and PRing the documentation, I can do that. I just don't want to step on anyone's toes.

[kevinpapst]

Totally forgot about your comment here, sorry @bkraul !

Yes, having multiple options is great. If you want to share it, please do so!
I think the community can only benefit from it.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. If you use Kimai on a daily basis, please consider donating to support further development of Kimai.