Skip to content

Commit fc3e27a

Browse files
committed
Introduce new DeleteConsecutiveAttributes maintenance procedure
In older versions of the OnTopic Library, new attribute versions could be saved even if their value had not changed. This isn't necessary in terms of the OnTopic library, which will take whatever the latest version of an attribute is when loading the topic graph (or restoring a topic). As such, these _consecutive duplicates_ only add clutter to the database, and potentially slow down some queries by introducing more data to sort through. The `DeleteConsecutiveAttributes` identifies and deletes these from the database, thus making it more efficient.
1 parent ba2094c commit fc3e27a

2 files changed

Lines changed: 70 additions & 0 deletions

File tree

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
--------------------------------------------------------------------------------------------------------------------------------
2+
-- DELETE CONSECUTIVE ATTRIBUTES
3+
--------------------------------------------------------------------------------------------------------------------------------
4+
-- Current versions of the OnTopic Library evaluate whether or not an attribute value has changed since the previous version,
5+
-- and doesn't create a new attribute version if it has. This wasn't true in previous versions, however. As a result, there are
6+
-- some cases, and especially in older databases, where unnecessary duplicates occur for attribute values. This script will
7+
-- detect concurrent duplicates and remove them from the database. This reduces the size of the database, without interfering
8+
-- with the data integrity. If attribute values are not consecutive, the duplicates aren't deleted; e.g., if the value of
9+
-- <c>Title</c> gets changed, then gets reverted, all three versions will be retained in the database.
10+
--------------------------------------------------------------------------------------------------------------------------------
11+
12+
CREATE PROCEDURE [dbo].[DeleteConsecutiveAttributes]
13+
AS
14+
15+
SET NOCOUNT ON;
16+
17+
--------------------------------------------------------------------------------------------------------------------------------
18+
-- CHECK INITIAL VALUES
19+
--------------------------------------------------------------------------------------------------------------------------------
20+
DECLARE @Count INT
21+
22+
SELECT @Count = Count(TopicID)
23+
FROM Attributes
24+
25+
Print('Initial Count: ' + CAST(@Count AS VARCHAR) + ' Attributes in the database.');
26+
27+
--------------------------------------------------------------------------------------------------------------------------------
28+
-- IDENTIFY GROUPS OF CONCURRENT DUPLICATES
29+
--------------------------------------------------------------------------------------------------------------------------------
30+
WITH GroupedValues AS (
31+
SELECT TopicID,
32+
AttributeKey,
33+
AttributeValue,
34+
DateModified,
35+
Version,
36+
ValueGroup = ROW_NUMBER() OVER(PARTITION BY TopicID, AttributeKey ORDER BY TopicID, AttributeKey, Version)
37+
- ROW_NUMBER() OVER(PARTITION BY TopicID, AttributeKey, AttributeValue ORDER BY TopicID, AttributeKey, Version)
38+
FROM Attributes
39+
),
40+
41+
--------------------------------------------------------------------------------------------------------------------------------
42+
-- RANK DUPLICATES BY DATE
43+
--------------------------------------------------------------------------------------------------------------------------------
44+
RankedValues AS (
45+
SELECT TopicID,
46+
AttributeKey,
47+
AttributeValue,
48+
DateModified,
49+
Version,
50+
ValueGroup,
51+
ValueRank = ROW_NUMBER() OVER(PARTITION BY ValueGroup, TopicID, AttributeKey, AttributeValue ORDER BY TopicID, AttributeKey, Version)
52+
FROM GroupedValues
53+
)
54+
55+
--------------------------------------------------------------------------------------------------------------------------------
56+
-- DELETE NEWER DUPLICATES
57+
--------------------------------------------------------------------------------------------------------------------------------
58+
DELETE
59+
FROM RankedValues
60+
WHERE ValueRank > 1;
61+
62+
--------------------------------------------------------------------------------------------------------------------------------
63+
-- CHECK FINAL VALUES
64+
--------------------------------------------------------------------------------------------------------------------------------
65+
SELECT @Count = @Count - Count(TopicID)
66+
FROM Attributes
67+
68+
Print('Final Count: ' + CAST(@Count AS VARCHAR) + ' Attributes were identified and deleted.')
69+

OnTopic.Data.Sql.Database/OnTopic.Data.Sql.Database.sqlproj

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -100,6 +100,7 @@
100100
<Build Include="Types\AttributeValues.sql" />
101101
<Build Include="Types\TopicList.sql" />
102102
<Build Include="Views\AttributeIndex.sql" />
103+
<Build Include="Maintenance\DeleteConsecutiveAttributes.sql" />
103104
</ItemGroup>
104105
<ItemGroup>
105106
<ArtifactReference Include="$(DacPacRootPath)\Extensions\Microsoft\SQLDB\Extensions\SqlServer\130\SqlSchemas\master.dacpac">

0 commit comments

Comments
 (0)