Skip to content

Commit f7786de

Browse files
JeremyCaneyJeremyCaney
committed
Introduce new ConsolidateVersions maintenance procedure
Currently, a version gets saved every time a value is changed. That potentially creates a lot of records over time—far more than the five historical versions we surface to the OnTopic editor. The resolution of those versions becomes impractical at some point: If a topic was edited ten times in a week five years ago, we don't likely care about each of those versions. The `ConsolidateVersions` stored procedure solves this by allowing _all_ versions within a given date range to be consolidated into a single, new version. This reduces the granularity. This can be used to consolidate all topics older than a specific date—or all topics _within_ a date _range_. The latter, for instance, could be used to compress each year down to a single version.
1 parent 1a2f1d6 commit f7786de

2 files changed

Lines changed: 108 additions & 0 deletions

File tree

‎OnTopic.Data.Sql.Database/Maintenance/ConsolidateVersions.sql‎

Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
--------------------------------------------------------------------------------------------------------------------------------
2+
-- CONSOLIDATE VERSIONS
3+
--------------------------------------------------------------------------------------------------------------------------------
4+
-- Given a start date and an end date, will consolidate all versions within that range into a single, new version. This has the
5+
-- benefit of reducing the number of versions in the database, reducing the database size, and making some database queries
6+
-- faster. If the end date parameter is not specified, it defaults to 2000-01-01, in which case ALL historical data will be
7+
-- collapsed prior to the start date.
8+
--------------------------------------------------------------------------------------------------------------------------------
9+
10+
CREATE PROCEDURE [dbo].[ConsolidateVersions]
11+
@StartDate datetime = 20000101,
12+
@EndDate datetime = null
13+
AS
14+
15+
--------------------------------------------------------------------------------------------------------------------------------
16+
-- DECLARE AND SET VARIABLES
17+
--------------------------------------------------------------------------------------------------------------------------------
18+
DECLARE @IsNestedTransaction BIT;
19+
20+
BEGIN TRY
21+
22+
--------------------------------------------------------------------------------------------------------------------------------
23+
-- BEGIN TRANSACTION
24+
--------------------------------------------------------------------------------------------------------------------------------
25+
IF (@@TRANCOUNT = 0)
26+
BEGIN
27+
SET @IsNestedTransaction = 0;
28+
BEGIN TRANSACTION;
29+
END
30+
ELSE
31+
BEGIN
32+
SET @IsNestedTransaction = 1;
33+
END
34+
35+
--------------------------------------------------------------------------------------------------------------------------------
36+
-- SELECT TOPIC ATTRIBUTES
37+
--------------------------------------------------------------------------------------------------------------------------------
38+
;WITH TopicAttributes
39+
AS (
40+
SELECT Version,
41+
RowNumber = ROW_NUMBER() OVER (
42+
PARTITION BY TopicID,
43+
AttributeKey
44+
ORDER BY Version DESC
45+
)
46+
FROM Attributes
47+
WHERE Version > @StartDate
48+
AND Version <= @EndDate
49+
)
50+
UPDATE TopicAttributes
51+
SET Version = @EndDate
52+
WHERE RowNumber = 1
53+
54+
--------------------------------------------------------------------------------------------------------------------------------
55+
-- DELETE CONSOLIDATED ATTRIBUTES
56+
--------------------------------------------------------------------------------------------------------------------------------
57+
DELETE
58+
FROM Attributes
59+
WHERE Version > @StartDate
60+
AND Version < @EndDate
61+
62+
--------------------------------------------------------------------------------------------------------------------------------
63+
-- SELECT EXTENDED ATTRIBUTES
64+
--------------------------------------------------------------------------------------------------------------------------------
65+
;WITH TopicExtendedAttributes
66+
AS (
67+
SELECT Version,
68+
RowNumber = ROW_NUMBER() OVER (
69+
PARTITION BY TopicID
70+
ORDER BY Version DESC
71+
)
72+
FROM ExtendedAttributes
73+
WHERE Version > @StartDate
74+
AND Version <= @EndDate
75+
)
76+
UPDATE TopicExtendedAttributes
77+
SET Version = @EndDate
78+
WHERE RowNumber = 1
79+
80+
--------------------------------------------------------------------------------------------------------------------------------
81+
-- DELETE CONSOLIDATED ATTRIBUTES
82+
--------------------------------------------------------------------------------------------------------------------------------
83+
DELETE
84+
FROM ExtendedAttributes
85+
WHERE Version > @StartDate
86+
AND Version < @EndDate
87+
88+
--------------------------------------------------------------------------------------------------------------------------------
89+
-- COMMIT TRANSACTION
90+
--------------------------------------------------------------------------------------------------------------------------------
91+
IF (@@TRANCOUNT > 0 AND @IsNestedTransaction = 0)
92+
BEGIN
93+
COMMIT
94+
END
95+
END TRY
96+
97+
--------------------------------------------------------------------------------------------------------------------------------
98+
-- HANDLE ERRORS
99+
--------------------------------------------------------------------------------------------------------------------------------
100+
BEGIN CATCH
101+
IF (@@TRANCOUNT > 0 AND @IsNestedTransaction = 0)
102+
BEGIN
103+
ROLLBACK;
104+
END;
105+
THROW
106+
RETURN;
107+
END CATCH

‎OnTopic.Data.Sql.Database/OnTopic.Data.Sql.Database.sqlproj‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -103,6 +103,7 @@
103103
<Build Include="Maintenance\DeleteConsecutiveExtendedAttributes.sql" />
104104
<Build Include="Maintenance\DeleteOrphanedLastModifiedAttributes.sql" />
105105
<Build Include="Maintenance\DeleteConsecutiveAttributes.sql" />
106+
<Build Include="Maintenance\ConsolidateVersions.sql" />
106107
</ItemGroup>
107108
<ItemGroup>
108109
<ArtifactReference Include="$(DacPacRootPath)\Extensions\Microsoft\SQLDB\Extensions\SqlServer\130\SqlSchemas\master.dacpac">

0 commit comments

Comments
 (0)