Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

dns: more keywords; plus some eve/keyword parity tooling - v8 #12664

Open
wants to merge 13 commits into
base: master
Choose a base branch
from
Open

dns: more keywords; plus some eve/keyword parity tooling - v8 #12664

wants to merge 13 commits into from
+1,021 −275

Conversation

jasonish
Copy link
Member

This PR builds on #12500 and also adds
the following keywords:

  • dns.additionals.rrname
  • dns.authorities.rrname

It builds on #12500 and it uses functions introduced as part of the work for
dns.response.rrname.

To provide consistency and better eve parity the following keywords were
renamed:

  • dns.query.rrname -> dns.queries.rrname
  • dns.answer.rrname -> dns.answers.rrname

And finally, as using DNS as a test protocol for parity, try some tooling for
eve parity.

For example, ./scripts/eve-parity.py mapped-keywords:

dns.rcode -> [dns.rcode]
dns.answer.additionals.rdata -> [dns.response.rrname]
dns.answer.additionals.rrname -> [dns.additionals.rrname, dns.response.rrname]
dns.answer.authorities.rdata -> [dns.response.rrname]
dns.answer.authorities.rrname -> [dns.authorities.rrname, dns.response.rrname]
dns.queries.rrname -> [dns.queries.rrname, dns.query]
dns.queries.rrtype -> [dns.rrtype]
dns.queries.opcode -> [dns.opcode]
dns.additionals.rdata -> [dns.response.rrname]
dns.additionals.rrname -> [dns.additionals.rrname, dns.response.rrname]
dns.authorities.rdata -> [dns.response.rrname]
dns.authorities.rrname -> [dns.authorities.rrname, dns.response.rrname]
dns.answers.rdata -> [dns.response.rrname]
dns.answers.rrname -> [dns.answers.rrname, dns.response.rrname]

and ./scripts/eve-parity.py unmapped-fields | grep dns:

dhcp.dns_servers
dns.aa
dns.additionals.opt.code
dns.additionals.opt.data
dns.additionals.rrtype
dns.additionals.ttl
dns.answer.additionals.opt.code
dns.answer.additionals.opt.data
dns.answer.additionals.rrtype
dns.answer.additionals.ttl
dns.answer.authorities.rdata_truncated
dns.answer.authorities.rrname_truncated
dns.answer.authorities.rrtype
dns.answer.authorities.soa.expire

and new in this version: unmapped-keywords to show known keywords that are
not mapped in the eve.json.

Other changes:

SV_BRANCH=OISF/suricata-verify#2311

scrivs86 and others added 13 commits February 21, 2025 16:11
@scrivs86 @jasonish
dns: add dns.response sticky buffer
15dd20d 
Feature: 7012
Add dns.response sticky buffer to match on dns response fields.
Add rust functions to return dns response packet data.
Unit tests verifying signature matching.
@scrivs86 @jasonish
doc/userguide: document dns.response
f58267e 
Feature: 7012
@jasonish
dns: rename dns.response keyword to dns.response.rrname
364057c 
This is a better name as the keyword is looking at all rrname type
fields in the response.
@jasonish
detect-dns-response: disable clang-format around byte arrays
5702306 
These arrays are manually formatted for readability.
@jasonish
dns: refactor function to get rrname to be safe
78e9c45 
Make the function safe by returning a reference to the DNSName object,
the unsafe C wrapper can do the conversion to pointers.
@jasonish
detect: split new keyword id from registration
b58ed81 
Split DetectHelperKeywordRegister into 2 functions, one for acquiring
a new keyword ID, and another to perform the registration.

This makes it easier to do the traditional C keyword initialization
with a dynamic ID.
@jasonish
dns: add keywords for additionals and authorities rrnames
3fe80bd 
Add keywords dns.additionals.rrname and dns.authorities.rrname. Along
the way, consolidate dns.query.name and dns.answer.name into a single file
and register them altogether since there is a lot of common code.
@jasonish
schema: add an object for mapping fields to keywords
76f5697 
To some EVE fields and a "suricata" object that contains an array of
keywords. These are the keywords that map directly to this field, or
somehow cover this field.

This is an attempt at tooling to help with EVE and keyword parity.

Related to tickets: OISF#5642, OISF#6463, OISF#4772
@jasonish
script/eve-parity: add script for checking eve/keyword parity
38f5fda 
Currently this script has two commands: "missing" and "having".

"missing" will show eve fields that do not map to any keywords.

"having" will sohw eve fields along with their keyword mappsings,
while also validating that those keywords really exist.

Related to tickets: OISF#6463, OISF#4772
@jasonish
detect-dns-response: remove unit tests
9ba7a28 
Should have coverage by S-V now.
@jasonish
schema: mark "stats" and "drop" as having no keywords
298262b
@jasonish
schema: mark dns.version and dns.grouped as having no keywords
41e2534
@jasonish
eve-parity: skip transform keywords
762e73e
@jasonish jasonish mentioned this pull request Feb 24, 2025
Closed
@suricata-qa
Copy link

Information: QA ran without warnings.

Pipeline 24877

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jufajardini jufajardini Awaiting requested review from jufajardini jufajardini is a code owner

@victorjulien victorjulien Awaiting requested review from victorjulien victorjulien is a code owner

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@jasonish @suricata-qa @scrivs86