A CA Notification Filter can be thought of as a search over the CEWL list. If the expression matches the criteria, then these rows are returned. When added to an alert, the alert is fired only when new CVEs match this condition. The search filter commands are 'packed-to-the-rafters' with features. We support the simplest search - a keyword search such as “remote code execution” as well as fully-blown filter expressions that have over 80 transformations. The 🔑 thing about filters is that they have to return a true or false value for every row, so think about it in these terms, what can you filter for or filter out based on some expression.
...
The CEWL (CVE Early Warning List) has a number of columns and definitions.
Column | Description | Type |
|---|---|---|
CVE (cve) | The CVE is usually the Mitre / NVD CVE. However, there are times where CTCI will create a CVE or prefix the CVE with a few letters such as R for Research, I for Intel, H for Honeypot, and p for provisional. An example would be HR-CVE-XXXX-XX, where we have a CVE we see that has matched our Honeypot and has been researched. | string |
Year (year_) | The year of the CVE, now we usually extract this from the CVE-YYYY-XXX, where YYYY is the year. However, there are a few CVEs that have been retrospective given different numbers. A year can be filtered and grouped on. | integer |
NVD Score (base_score) | This is the NVD base score for this CVE. For unknown values, sometimes this is set to Unknown or sometimes set to a value. When it is, we will add that to the External note. | float |
CVE Date (cve_date) | This was when the CVE was published to Mitre, which may or may not had had any details or scoring when it was published. | the string then converted to a date-time object in the back end |
NVD Severity (base_severity) | This is the Severity that is found in NVD. When it is an unknown value, we sometimes then set this value. If we do, then this is added to that External note. | string |
Vendor (vendor) | This is the vendor; if there is a large number of CPEs for vendors, we still take what Mitre has published. Sometimes for consistency, we will change the capitalization of the Vendor. | string |
Product (product) | This is the product; if there is a large number of CPEs for vendors, we still take what Mitre has published as the base product. Sometimes for consistency, we will change the capitalization of the product. | string |
Vulnerability Type (vulnerability_type) | This is scarcely added, as you can get some of this information from the CWE. | string |
Description (description) | This is mostly the NVD/Mitre description of the CVE. | string / note |
Date Added (date_added) | That is when this was added to the list. | string, then converted to date time on the backend |
Reference (reference) | This is mostly the NVD reference unless it’s not easily found or there is a lot better reference. | string |
CVSSv2 | This is a different scoring system, so it makes sense to use CVSSv2 | boolean |
CVSSv3 | This is for CVEs greater than 2016. | boolean |
CVSSv4 | Future - we will have this option as well | boolean |
id_ref | This is a monotonic increasing number used by API and other services to know when we have new CVEs that have to be notified. | integer |
Updated On (updated_on) | Today this isn’t a good indicator, as we are generating the updates and checks to the list daily, and thus the field is current. In the future, we want this to be exactly that - updated on. | string, then converted to date time on the backend |
CWE (we) | The standard for Common Weakness Enumeration (CWE), where possible we try to retrofit a |
CWE if one is not properly found. | string | |
Honeypot (honeypot) | This is the date/time within reason on where we saw the first hit of this CVE exploit. | string, then on the back-end is converted to a date-time object |
Intel (Intel) | This is the date/time within reason on where we saw the automated intel hit of this CVE exploit. | string, then on the back-end is converted to a date-time object |
Research (research) | This is the date/time within reason on where we an analysts did manual research to find the first hit of this CVE exploit | string, then on the back-end is converted to a date-time object |
Processed | This is a flag to say whether you have processed this CVE entry. It’s helpful for integrating what you have already processed and not. Within the UI, you can show all entries, processed entries as grayed out and hide the processed entries. | boolean |
Filter expression language
...
Most commonly used string operations.
Operation (<column name>.str.<operation>) | Details |
|---|---|
isnull() | If empty |
not <column name>.str.isnull() | If not empty. If it has something in it! |
contains(“<string>”) | If it contains the <string>, for example description.str.contains(“remote code execution”) where return all the CEWL entries where the description contains remote code execution. Contains also supports regex - way cool! |
startswith(“<string>”) | Suppose the entry for the column starts with <string>, for example, where vendor.str.startswith(“Palo”), to return all the vendors that start with Palo - should return Palo Alto Networks. |
endswith(“<string>”) | If the entry for the column ends with <string>, for example, where product.str.endswith(“Player”), to return all the products that end with Player, would return things like Flash Player and such. |
len() | Returns the length of the entry, so you could find all products where the length of the product name is less than 5, product.str.len()<=5 |
slice() | slice(start, stop) |
isnum() | Is the string a number |
isalphanum() | Is the string alphanumeric, numbers, and letters |
cat() | Concatenate a string with a string |
get() | Get a character at the index value. Indexes start at zero. |
join() | Join a string with a string. |
split() | Split a string on a delimiter |
rsplit() | Split strings on delimiter working from the end of the string |
replace() | Replace occurrences of pattern/regex/string with some other string or the return value of a callable given the occurrence |
strip() | Strips spaces or whatever value you pass it |
rstrip() | Strips spaces or whatever value you pass it from the right-hand side |
lstrip() | Strips spaces or whatever value you pass it from the left-hand side |
Other string operations
Operation (<column name>.str.<operation>) | Details |
|---|---|
repeat() | Duplicate values (<column>.str.repeat(3) equivalent to x * 3) |
pad() | Add whitespace to left, right, or both sides of strings |
zfill() | Pad strings in the Series/Index by prepending ‘0’ characters. |
slice_replace() | Replace slice in each string with the passed value |
count() | Count occurrences of pattern |
findall() | Compute a list of all occurrences of pattern/regex for each string |
match() | Call re.match on each element, returning matched groups as a list |
extract() | Call re.search on each element, returning DataFrame with one row for each element and one column for each regex capture group |
extractall() | Call re.findall on each element, returning DataFrame with one row for each match and one column for each regex capture group |
lower() | Make the string lower |
casefold() | Make the string have a case fold |
upper() | Make the string upper case |
find() | Return the index of a string find |
capitalize() | Return the string capitalized |
swapcase() | Return the string with the case swapped |
isalnum() | Returns whether all characters are alphanumeric |
isalpha() | Returns whether all characters are alphabetic |
isdigit() | Returns whether all characters are digits |
isspace() | Returns whether all characters are spaces |
islower() | Returns whether the string is lower |
isupper() | Returns whether the string is upper |
istitle() | Returns whether the string is in title case |
isnumeric() | Returns whether the string is numeric |
isdecimal() | Returns whether the string is decimal |
There are even more at https://pandas.pydata.org/docs/index.html
...
The Id is the internal reference to this Notification Filter. The Save button is self-explanatory. The back button goes back to the list of Notification Filters.
Field Name | Details |
|---|---|
Id | Ignore - internal reference, and you can ignore |
Filter Name | This is required and is the name you would like to give to the Notification Filter. |
Enabled | Whether this Notification Filter is Enabled, it’s disabled by default. |
Note | This is a multi-line field that you can leave a note for someone about this Notification Filter. Putting who to contact, who owns this Notification Filter, and other details could really help other people and new hires. |
Filter | This is the filter. If the filter is empty, it will send all new CVEs to the Notification Delivery or to the default Email Address. |
Company Name | This will be your Company Name by default if you don’t set it. If you manage Partners or other companies, you can select where this Notification Group gets created. |
Notification Group | A filter needs to belong to a Notification Group |
Notification Email | This is a default email value, and you don’t have to set a Delivery Group |
Notification Delivery | This is a Notification Delivery such as Email, Slack, Teams, Jira, and Webhook. |
Notification Filter Role | Future - this will allow you to have different people editing this Notification Filter. |
Notification Filter Expiry | Future - To be able to expire this Notification Delivery after a certain amount of time has expired. |
Notification Filter Permission | Future - this will allow you to have different people editing this Notification Delivery. |
Notification Schedule | Future - To allow to run filters at different times. Currently, they run every time there is a new CVE created and then notified straight away. |
How to Edit a Notification Filters
...