Caching in Docker - the hardest thing in computer science - Comment
|
Presentation about problems encountered while building Breeze - Development Environment for Apache Airflow. Docker is great for containerisation but when you are trying to make full use of it for caching and development, there are a number of problems you have to overcome. Topics covered in this presentation slides: 1. The hardest thing in computer science 2. Hard things 3. Docker Caching Dependency versions Install dependencies [ 20 minutes or so ] Only here copy all sources 4. Intended behaviour ● No change: docker is not rebuilt - LIGHTNING FAST!!!! ● Sources change/dependencies not: only sources are added - QUITE FAST !!! ● Dependencies change: dependencies installed, sources - LITTLE SLOWER !! 5. Actual behaviour same machine - local checkout ● Local docker registry ● Repeated build: 1:06m ● Only sources: 1:30m ● Dependencies: 11m ● Whole build: ~ 30m 6. CI case ● Always fresh machine ○ no code ○ no registry ● Git clone/checkout ● Build ● Wipeout 7. Docker registry to the rescue! Build cache: ● Docker build ● Docker push airflow/airflow:latest Use cache: ● Docker pull airflow/airflow:latest ● docker build --cache-from ariflow/airflow:latest 8. Actual behaviour Docker Hub automated build ● DockerHub docker registry as cache ● Repeated build: 11m ● Only sources: 11m |
|||
| Posted by : peter88 | Post date : 2020-01-06 14:37 | ||
| Category : Technology | Views : 376 | ||
New Comment
