Scaraplate Template¶
Scaraplate uses cookiecutter under the hood, so the scaraplate template is a cookiecutter template with the following properties:
- There must be a
scaraplate.yaml
config in the root of the template dir (near thecookiecutter.json
). - The template dir must be a git repo (because some strategies might render URLs to the template project and HEAD commit, making it easy to find out what template was used to rollup, see Template Git Remotes).
- The cookiecutter’s project dir must be called
project_dest
, i.e. the template must reside in the{{cookiecutter.project_dest}}
directory. - The template must contain a file which renders the current cookiecutter context. Scaraplate then reads that context to re-apply cookiecutter template on subsequent rollups (see Cookiecutter Context Types).
scaraplate.yaml
contains:
- strategies (see Strategies),
- cookiecutter context type (see Cookiecutter Context Types),
- template git remote (see Template Git Remotes).
Note
Neither scaraplate.yaml
nor cookiecutter.json
would get
to the target project. These two files exist only in the template
repo. The files that would get to the target project are located
in the inner {{cookiecutter.project_dest}}
directory of the template repo.
scaraplate rollup
has a --no-input
switch which doesn’t ask for
cookiecutter context values. This can be used to automate rollups
when the cookiecutter context is already present in the target project
(i.e. scaraplate rollup
has been applied before). But the first rollup
should be done without the --no-input
option, so the cookiecutter
context values could be filled by hand interactively.
The arguments to the scaraplate rollup
command must be local
directories (i.e. the template git repo must be cloned manually,
scaraplate doesn’t support retrieving templates from git remote directly).
Scaraplate Example Template¶
We maintain an example template for a new Python project here: https://github.com/rambler-digital-solutions/scaraplate-example-template
You may use it as a starting point for creating your own scaraplate template. Of course it doesn’t have to be for a Python project: the cookiecutter template might be for anything. A Python project is just an example.
Creating a new project from the template¶
$ git clone https://github.com/rambler-digital-solutions/scaraplate-example-template.git
$ scaraplate rollup ./scaraplate-example-template ./myproject
`myproject1/.scaraplate.conf` file doesn't exist, continuing with an empty context...
`project_dest` must equal to "myproject"
project_dest [myproject]:
project_monorepo_name []:
python_package [myproject]:
metadata_name [myproject]:
metadata_author: Kostya Esmukov
metadata_author_email: kostya@esmukov.net
metadata_description: My example project
metadata_long_description [file: README.md]:
metadata_url [https://github.com/rambler-digital-solutions/myproject]:
coverage_fail_under [100]: 90
mypy_enabled [1]:
Done!
$ tree -a myproject
myproject
├── .editorconfig
├── .gitignore
├── .scaraplate.conf
├── MANIFEST.in
├── Makefile
├── README.md
├── mypy.ini
├── setup.cfg
├── setup.py
├── src
│ └── myproject
│ └── __init__.py
└── tests
├── __init__.py
└── test_metadata.py
3 directories, 12 files
The example template also contains a project_monorepo_name
variable
which simplifies creating subprojects in monorepos (e.g. a single git
repository for multiple projects). In this case scaraplate should be
applied to the inner projects:
$ scaraplate rollup ./scaraplate-example-template ./mymonorepo/innerproject
`mymonorepo/innerproject/.scaraplate.conf` file doesn't exist, continuing with an empty context...
`project_dest` must equal to "innerproject"
project_dest [innerproject]:
project_monorepo_name []: mymonorepo
python_package [mymonorepo_innerproject]:
metadata_name [mymonorepo-innerproject]:
metadata_author: Kostya Esmukov
metadata_author_email: kostya@esmukov.net
metadata_description: My example project in a monorepo
metadata_long_description [file: README.md]:
metadata_url [https://github.com/rambler-digital-solutions/mymonorepo]:
coverage_fail_under [100]: 90
mypy_enabled [1]:
Done!
$ tree -a mymonorepo
mymonorepo
└── innerproject
├── .editorconfig
├── .gitignore
├── .scaraplate.conf
├── MANIFEST.in
├── Makefile
├── README.md
├── mypy.ini
├── setup.cfg
├── setup.py
├── src
│ └── mymonorepo_innerproject
│ └── __init__.py
└── tests
├── __init__.py
└── test_metadata.py
4 directories, 12 files
Updating a project from the template¶
$ scaraplate rollup ./scaraplate-example-template ./myproject --no-input
Continuing with the following context from the `myproject/.scaraplate.conf` file:
{'_template': 'scaraplate-example-template',
'coverage_fail_under': '90',
'metadata_author': 'Kostya Esmukov',
'metadata_author_email': 'kostya@esmukov.net',
'metadata_description': 'My example project',
'metadata_long_description': 'file: README.md',
'metadata_name': 'myproject',
'metadata_url': 'https://github.com/rambler-digital-solutions/myproject',
'mypy_enabled': '1',
'project_dest': 'myproject',
'project_monorepo_name': '',
'python_package': 'myproject'}
Done!
Cookiecutter Context Types¶
cookiecutter context are the variables specified in cookiecutter.json
,
which should be provided to cookiecutter to cut a project from the template.
The context should be generated by one of the files in the template, so scaraplate could read these variables and rollup the template automatically (i.e. without asking for these variables).
The default context reader is ScaraplateConf
, but a custom one
might be specified in scaraplate.yaml
like this:
cookiecutter_context_type: scaraplate.cookiecutter.SetupCfg
Bases:
abc.ABC
The abstract base class for retrieving cookiecutter context from the target project.
This class can be extended to provide a custom implementation of the context reader.
Init the context reader.
Retrieve the context.
If the target file doesn’t exist,
FileNotFoundError
must be raised.If the file doesn’t contain the context, an empty dict should be returned.
Built-in Cookiecutter Context Types¶
Bases:
scaraplate.cookiecutter.CookieCutterContext
A default context reader which assumes that the cookiecutter template contains the following file named
.scaraplate.conf
in the root of the project dir:[cookiecutter_context] {%- for key, value in cookiecutter.items()|sort %} {%- if key not in ('_output_dir',) %} {{ key }} = {{ value }} {%- endif %} {%- endfor %}
Cookiecutter context would be rendered in the target project by this file, and this class is able to retrieve that context from it.
Bases:
scaraplate.cookiecutter.CookieCutterContext
A context reader which retrieves the cookiecutter context from a section in
setup.cfg
file.The
setup.cfg
file must be in the cookiecutter template and must contain the following section:[tool:cookiecutter_context] {%- for key, value in cookiecutter.items()|sort %} {%- if key not in ('_output_dir',) %} {{ key }} = {{ value }} {%- endif %} {%- endfor %}
Template Maintenance¶
Given that scaraplate provides ability to update the already created projects from the updated templates, it’s worth discussing the maintenance of a scaraplate template.
Removing a template variable¶
Template variables could be used as feature flags to gradually introduce some changes in the templates which some target projects might not use (yet) by disabling the flag.
But once the migration is complete, you might want to remove the no longer needed variable.
Fortunately this is very simple: just stop using it in the template and
remove it from cookiecutter.json
. On the next scaraplate rollup
the removed variable will be automatically removed from
the cookiecutter context file.
Adding a new template variable¶
The process for adding a new variable is the same as for removing one:
just add it to the cookiecutter.json
and you can start using it in
the template.
If the next scaraplate rollup
is run with --no-input
, the new
variable will have the default value as specified in cookiecutter.json
.
If you need a different value, you have 2 options:
- Run
scraplate rollup
without the--no-input
flag so the value for the new variable could be asked interactively. - Manually add the value to
the cookiecutter context section
so the next
rollup
could pick it up.
Restructuring files¶
Scaraplate strategies intentionally don’t provide support for anything more complex than a simple file-to-file change. It means that a scaraplate template cannot:
- Delete or move files in the target project;
- Take multiple files and union them.
The reason is simple: such operations are always the one-time ones so it is just easier to perform them manually once than to maintain that logic in the template.
Patterns¶
This section contains some patterns which might be helpful for creating and maintaining a scaraplate template.
Feature flags¶
Let’s say you have a template which you have applied to dozens of your projects.
And now you want to start gradually introducing a new feature, let it be a new linter.
You probably would not want to start using the new thing everywhere at once. Instead, usually you start with one or two projects, gain experience and then start rolling it up on the other projects.
For that you can use template variables as feature flags.
The example template contains
a mypy_enabled
variable which demonstrates this concept. Basically
it is a regular cookiecutter variable, which can take different values
in the target projects and thus affect the template by enabling or disabling
the new feature.
Include files¶
Consider Makefile
. On one hand, you would definitely want to have some
make targets to come from the template; on the other hand, you might need
to introduce custom make targets in some projects. Coming up with a scaraplate
strategy which could merge such a file would be quite difficult.
Fortunately, Makefile
allows to include other files. So the solution
is quite trivial: have Makefile
synced from the template (with
the scaraplate.strategies.Overwrite
strategy), and include
a Makefile.inc
file from there which will not be overwritten by the template.
This concept is demonstrated in the example template.
Manual merging¶
Sometimes you need to merge some files which might be modified in the target
projects and for which there’s no suitable strategy yet. In this case
you can use scaraplate.strategies.TemplateHash
strategy as
a temporary solution: it would overwrite the file each time a new
git commit is added to the template, but keep the file unchanged since
the last rollup of the same template commit.
The example template uses this
approach for setup.py
.
Create files conditionally¶
Cookiecutter hooks can be used to post-process the generated temporary project. For example, you might want to skip some files from the template depending on the variables.
The example template contains
an example hook which deletes mypy.ini
file when the mypy_enabled
variable is not set to 1
.