Skip to content

Allow for custom log formats to be defined in stacker.yaml#705

Merged
russellballestrini merged 1 commit into
cloudtools:masterfrom
russellballestrini:rb-custom-log-formats
Mar 6, 2019
Merged

Allow for custom log formats to be defined in stacker.yaml#705
russellballestrini merged 1 commit into
cloudtools:masterfrom
russellballestrini:rb-custom-log-formats

Conversation

@russellballestrini

@russellballestrini russellballestrini commented Mar 4, 2019

Copy link
Copy Markdown
Member

At Remind we run stacker on a large portion of our stacks in stage and production concurrently using continuous delivery. This means our logs are interlaced between stage and prod and are hard to use.

Also since we use continuous delivery we want to put the deployment_id or git commit hash on every log line so that we can aggregate them all up with our log parsing and viewing tools.

This PR makes it possible to set a custom log format from stacker.yaml.

modified:   .gitignore
modified:   stacker/commands/stacker/__init__.py
modified:   stacker/commands/stacker/base.py
modified:   stacker/config/__init__.py
modified:   stacker/logger/__init__.py
new file:   stacker/tests/fixtures/not-basic.env
new file:   stacker/tests/fixtures/vpc-custom-log-format-info.yaml
modified:   stacker/tests/test_stacker.py

@ejholmes ejholmes left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems pretty reasonable to me, and seems pretty common through the python world to be able to do this.

I'll let @phobologic do the final review, but I'm 👍

@@ -0,0 +1,24 @@
log_format_info: "[%(asctime)s] ${environment} custom log format - %(message)s"

@ejholmes ejholmes Mar 5, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In stacker.logger.setup_logging we also support color and debug keys. Should we just make this a nested structure?

Example

log_format:
  info: "[%(asctime)s] ${environment} custom log format - %(message)s"
  color: # Some color format
  debug: # Some debug format

@russellballestrini russellballestrini changed the title Allow for custom log formats to be defined in stacks.yaml Allow for custom log formats to be defined in stacker.yaml Mar 5, 2019

@phobologic phobologic left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me - a few minor comments, and I wonder if we could have tests that actually test that the log formats are changing. Not sure that's super important though.

Comment thread .gitignore

tests/fixtures/blueprints/*-result

FakeKey.pem

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's this about?

@russellballestrini russellballestrini Mar 6, 2019

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some other new test added testing for keys (#700), and it doesn't seem to clean up after itself. Since I run tests locally they show up in my repo after running make test.

Comment thread docs/config.rst Outdated

You may optionally provide a custom log format for stacker to use::

# optional log format overrides.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be worth putting the defaults here, or at least explaining how we use the python logging module standard format variables (link to here? https://docs.python.org/2.7/library/logging.html#logrecord-attributes)

Docs and config model adjustment.

	modified:   .gitignore
	modified:   docs/config.rst
	modified:   stacker/commands/stacker/__init__.py
	modified:   stacker/commands/stacker/base.py
	modified:   stacker/config/__init__.py
	modified:   stacker/logger/__init__.py
	new file:   stacker/tests/fixtures/not-basic.env
	new file:   stacker/tests/fixtures/vpc-custom-log-format-info.yaml
	modified:   stacker/tests/test_stacker.py
@russellballestrini russellballestrini merged commit 3f97c9f into cloudtools:master Mar 6, 2019
@russellballestrini russellballestrini deleted the rb-custom-log-formats branch March 6, 2019 14:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants