Skip to content

Rust: Reduce CI noise from the SummaryStats query #19180

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
geoffw0 merged 11 commits into from
Apr 4, 2025
Merged

Rust: Reduce CI noise from the SummaryStats query #19180

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
ff2a1ca
Rust: Group the data in rust/summary/summary-statistics.
geoffw0 Mar 11, 2025
32e2c19
Rust: Move all summary stats logic into Stats.qll.
geoffw0 Mar 11, 2025
cc90ba5
Rust: Create a less noisy version of the summary stats query, for use…
geoffw0 Mar 11, 2025
86840a7
Rust: Update .expected files.
geoffw0 Mar 11, 2025
9fc0ee1
Rust: Change the query ID to rust/summary/summary-statistics-reduced.
geoffw0 Apr 2, 2025
02245af
Rust: Rename the query file.
geoffw0 Apr 2, 2025
fbde235
Rust: Rename the test as well.
geoffw0 Apr 2, 2025
dc31da8
Rust: Fix file name typo.
geoffw0 Apr 4, 2025
e9971ff
Rust: Change the ID of the new query.
geoffw0 Apr 4, 2025
64ee6c5
Merge branch 'main' into lessnoise
geoffw0 Apr 4, 2025
6597454
Merge remote-tracking branch 'upstream/main' into lessnoise
geoffw0 Apr 4, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 0 additions & 10 deletions rust/ql/integration-tests/hello-project/summary.expected
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
| Elements extracted | 67 |
| Elements unextracted | 0 |
| Extraction errors | 0 |
| Extraction warnings | 1 |
| Files extracted - total | 5 |
Expand All @@ -15,11 +13,3 @@
| Macro calls - resolved | 2 |
| Macro calls - total | 2 |
| Macro calls - unresolved | 0 |
| Taint edges - number of edges | 1674 |
| Taint reach - nodes tainted | 0 |
| Taint reach - per million nodes | 0 |
| Taint sinks - cryptographic operations | 0 |
| Taint sinks - query sinks | 1 |
| Taint sources - active | 0 |
| Taint sources - disabled | 0 |
| Taint sources - sensitive data | 0 |
2 changes: 1 addition & 1 deletion rust/ql/integration-tests/hello-project/summary.qlref
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1 @@
queries/summary/SummaryStats.ql
queries/summary/SummaryStatsLess.ql
Comment thread Fixed
10 changes: 0 additions & 10 deletions rust/ql/integration-tests/hello-workspace/summary.cargo.expected
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
| Elements extracted | 90 |
| Elements unextracted | 0 |
| Extraction errors | 0 |
| Extraction warnings | 0 |
| Files extracted - total | 4 |
Expand All @@ -15,11 +13,3 @@
| Macro calls - resolved | 2 |
| Macro calls - total | 2 |
| Macro calls - unresolved | 0 |
| Taint edges - number of edges | 1674 |
| Taint reach - nodes tainted | 0 |
| Taint reach - per million nodes | 0 |
| Taint sinks - cryptographic operations | 0 |
| Taint sinks - query sinks | 1 |
| Taint sources - active | 0 |
| Taint sources - disabled | 0 |
| Taint sources - sensitive data | 0 |
2 changes: 1 addition & 1 deletion rust/ql/integration-tests/hello-workspace/summary.qlref
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1 @@
queries/summary/SummaryStats.ql
queries/summary/SummaryStatsLess.ql
Comment thread Fixed
10 changes: 0 additions & 10 deletions rust/ql/integration-tests/hello-workspace/summary.rust-project.expected
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
| Elements extracted | 90 |
| Elements unextracted | 0 |
| Extraction errors | 0 |
| Extraction warnings | 0 |
| Files extracted - total | 4 |
Expand All @@ -15,11 +13,3 @@
| Macro calls - resolved | 2 |
| Macro calls - total | 2 |
| Macro calls - unresolved | 0 |
| Taint edges - number of edges | 1674 |
| Taint reach - nodes tainted | 0 |
| Taint reach - per million nodes | 0 |
| Taint sinks - cryptographic operations | 0 |
| Taint sinks - query sinks | 1 |
| Taint sources - active | 0 |
| Taint sources - disabled | 0 |
| Taint sources - sensitive data | 0 |
92 changes: 92 additions & 0 deletions rust/ql/src/queries/summary/Stats.qll
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,9 @@ private import codeql.rust.internal.PathResolutionConsistency as PathResolutionC
private import codeql.rust.controlflow.internal.CfgConsistency as CfgConsistency
private import codeql.rust.dataflow.internal.DataFlowConsistency as DataFlowConsistency
private import codeql.rust.Concepts
private import codeql.rust.Diagnostics
private import codeql.rust.security.SensitiveData
private import TaintReach
// import all query extensions files, so that all extensions of `QuerySink` are found
private import codeql.rust.security.CleartextLoggingExtensions
private import codeql.rust.security.SqlInjectionExtensions
Expand Down Expand Up @@ -72,3 +75,92 @@ int getTaintEdgesCount() {
* Gets a count of the total number of query sinks in the database.
*/
int getQuerySinksCount() { result = count(QuerySink s) }

class CrateElement extends Element {
CrateElement() {
this instanceof Crate or
this instanceof NamedCrate or
this.(AstNode).getParentNode*() = any(Crate c).getModule()
}
}

/**
* Gets summary statistics about individual elements in the database.
*/
predicate elementStats(string key, int value) {
key = "Elements extracted" and
value = count(Element e | not e instanceof Unextracted and not e instanceof CrateElement)
or
key = "Elements unextracted" and value = count(Unextracted e)
}

/**
* Gets summary statistics about extraction.
*/
predicate extractionStats(string key, int value) {
key = "Extraction errors" and value = count(ExtractionError e)
or
key = "Extraction warnings" and value = count(ExtractionWarning w)
or
key = "Files extracted - total" and value = count(ExtractedFile f | exists(f.getRelativePath()))
or
key = "Files extracted - with errors" and
value =
count(ExtractedFile f |
exists(f.getRelativePath()) and not f instanceof SuccessfullyExtractedFile
)
or
key = "Files extracted - without errors" and
value = count(SuccessfullyExtractedFile f | exists(f.getRelativePath()))
or
key = "Files extracted - without errors %" and
value =
(count(SuccessfullyExtractedFile f | exists(f.getRelativePath())) * 100) /
count(ExtractedFile f | exists(f.getRelativePath()))
or
key = "Lines of code extracted" and value = getLinesOfCode()
or
key = "Lines of user code extracted" and value = getLinesOfUserCode()
or
key = "Macro calls - total" and value = count(MacroCall mc)
or
key = "Macro calls - resolved" and value = count(MacroCall mc | mc.hasExpanded())
or
key = "Macro calls - unresolved" and value = count(MacroCall mc | not mc.hasExpanded())
}

/**
* Gets summary statistics about inconsistencies.
*/
predicate inconsistencyStats(string key, int value) {
key = "Inconsistencies - AST" and value = getTotalAstInconsistencies()
or
key = "Inconsistencies - Path resolution" and value = getTotalPathResolutionInconsistencies()
or
key = "Inconsistencies - CFG" and value = getTotalCfgInconsistencies()
or
key = "Inconsistencies - data flow" and value = getTotalDataFlowInconsistencies()
}

/**
* Gets summary statistics about taint.
*/
predicate taintStats(string key, int value) {
key = "Taint sources - active" and value = count(ActiveThreatModelSource s)
or
key = "Taint sources - disabled" and
value = count(ThreatModelSource s | not s instanceof ActiveThreatModelSource)
or
key = "Taint sources - sensitive data" and value = count(SensitiveData d)
or
key = "Taint edges - number of edges" and value = getTaintEdgesCount()
or
key = "Taint reach - nodes tainted" and value = getTaintedNodesCount()
or
key = "Taint reach - per million nodes" and value = getTaintReach().floor()
or
key = "Taint sinks - query sinks" and value = getQuerySinksCount()
or
key = "Taint sinks - cryptographic operations" and
value = count(Cryptography::CryptographicOperation o)
}
74 changes: 4 additions & 70 deletions rust/ql/src/queries/summary/SummaryStats.ql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,81 +7,15 @@
*/

import rust
import codeql.rust.Concepts
import codeql.rust.security.SensitiveData
import codeql.rust.security.WeakSensitiveDataHashingExtensions
import codeql.rust.Diagnostics
import Stats
import TaintReach

class CrateElement extends Element {
CrateElement() {
this instanceof Crate or
this instanceof NamedCrate or
this.(AstNode).getParentNode*() = any(Crate c).getModule()
}
}

from string key, int value
where
key = "Elements extracted" and
value = count(Element e | not e instanceof Unextracted and not e instanceof CrateElement)
or
key = "Elements unextracted" and value = count(Unextracted e)
or
key = "Extraction errors" and value = count(ExtractionError e)
or
key = "Extraction warnings" and value = count(ExtractionWarning w)
or
key = "Files extracted - total" and value = count(ExtractedFile f | exists(f.getRelativePath()))
or
key = "Files extracted - with errors" and
value =
count(ExtractedFile f |
exists(f.getRelativePath()) and not f instanceof SuccessfullyExtractedFile
)
or
key = "Files extracted - without errors" and
value = count(SuccessfullyExtractedFile f | exists(f.getRelativePath()))
or
key = "Files extracted - without errors %" and
value =
(count(SuccessfullyExtractedFile f | exists(f.getRelativePath())) * 100) /
count(ExtractedFile f | exists(f.getRelativePath()))
or
key = "Lines of code extracted" and value = getLinesOfCode()
or
key = "Lines of user code extracted" and value = getLinesOfUserCode()
or
key = "Inconsistencies - AST" and value = getTotalAstInconsistencies()
or
key = "Inconsistencies - Path resolution" and value = getTotalPathResolutionInconsistencies()
or
key = "Inconsistencies - CFG" and value = getTotalCfgInconsistencies()
or
key = "Inconsistencies - data flow" and value = getTotalDataFlowInconsistencies()
or
key = "Macro calls - total" and value = count(MacroCall mc)
or
key = "Macro calls - resolved" and value = count(MacroCall mc | mc.hasExpanded())
or
key = "Macro calls - unresolved" and value = count(MacroCall mc | not mc.hasExpanded())
or
key = "Taint sources - active" and value = count(ActiveThreatModelSource s)
or
key = "Taint sources - disabled" and
value = count(ThreatModelSource s | not s instanceof ActiveThreatModelSource)
or
key = "Taint sources - sensitive data" and value = count(SensitiveData d)
or
key = "Taint edges - number of edges" and value = getTaintEdgesCount()
or
key = "Taint reach - nodes tainted" and value = getTaintedNodesCount()
elementStats(key, value)
or
key = "Taint reach - per million nodes" and value = getTaintReach().floor()
extractionStats(key, value)
or
key = "Taint sinks - query sinks" and value = getQuerySinksCount()
inconsistencyStats(key, value)
or
key = "Taint sinks - cryptographic operations" and
value = count(Cryptography::CryptographicOperation o)
taintStats(key, value)
select key, value order by key
18 changes: 18 additions & 0 deletions rust/ql/src/queries/summary/SummaryStatsLess.ql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
/**
* @name Summary Statistics Less
* @description A table of summary statistics about a database, with data that
* has been found to be noisy on tests removed.
* @kind metric
* @id rust/summary/summary-statistics-less
Copy link
Copy Markdown
Contributor

@hvitved hvitved Apr 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SummeryStatsReduced.ql and ust/summary/summary-statistics-reduced instead?

Copy link
Copy Markdown
Contributor Author

@geoffw0 geoffw0 Apr 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I think I like that a bit more. Updated.

* @tags summary
*/

import rust
import Stats

from string key, int value
where
extractionStats(key, value)
or
inconsistencyStats(key, value)
select key, value order by key
1 change: 0 additions & 1 deletion rust/ql/test/query-tests/diagnostics/SummaryStats.qlref
View file
Open in desktop

This file was deleted.

10 changes: 0 additions & 10 deletions ...y-tests/diagnostics/SummaryStats.expected → ...sts/diagnostics/SummaryStatsLess.expected
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
| Elements extracted | 406 |
| Elements unextracted | 0 |
| Extraction errors | 0 |
| Extraction warnings | 7 |
| Files extracted - total | 7 |
Expand All @@ -15,11 +13,3 @@
| Macro calls - resolved | 8 |
| Macro calls - total | 9 |
| Macro calls - unresolved | 1 |
| Taint edges - number of edges | 1674 |
| Taint reach - nodes tainted | 0 |
| Taint reach - per million nodes | 0 |
| Taint sinks - cryptographic operations | 0 |
| Taint sinks - query sinks | 3 |
| Taint sources - active | 0 |
| Taint sources - disabled | 0 |
| Taint sources - sensitive data | 0 |
1 change: 1 addition & 0 deletions rust/ql/test/query-tests/diagnostics/SummaryStatsLess.qlref
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
queries/summary/SummaryStatsLess.ql
Comment thread Fixed