Skip to content

Commit a430fc9

Browse files
acoburnjholleran
acoburn
and
jholleran
authored
Add a retry mechanism for authorization tests (#118)
* Add retries to authorization tests --------- Co-authored-by: Jarlath Holleran <jarlathh@inrupt.com>
1 parent 6fe7818 commit a430fc9

7 files changed

Lines changed: 38 additions & 31 deletions

File tree

‎web-access-control/protected-operation/common.feature‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,12 @@ Scenario:
99
return agentLowerCase !== 'public' ? clients[agentLowerCase].getAuthHeaders(method, url) : {}
1010
}
1111
"""
12+
* def includesExpectedStatus =
13+
"""
14+
function (actual, expected) {
15+
return expected.includes(actual);
16+
}
17+
"""
1218
* def getRequestData =
1319
"""
1420
function (type) {

‎web-access-control/protected-operation/read-access-agent.feature‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -18,8 +18,8 @@ Feature: Only authenticated agents can read (and only that) a resource when gran
1818
* def testResource = utils.testResources[utils.getResourceKey(container, resource, type)]
1919
Given url testResource.url
2020
And headers utils.authHeaders(method, testResource.url, agent)
21+
And retry until responseStatus == <status>
2122
When method <method>
22-
Then status <status>
2323
Examples:
2424
| agent | result | method | type | container | resource | status |
2525
| Bob | can | GET | plain | no | R | 200 |
@@ -57,8 +57,8 @@ Feature: Only authenticated agents can read (and only that) a resource when gran
5757
And headers utils.authHeaders(method, testResource.url, agent)
5858
And header Content-Type = 'text/turtle'
5959
And request '@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#>. <> rdfs:comment "Bob added this.".'
60+
And retry until utils.includesExpectedStatus(responseStatus, <status>)
6061
When method <method>
61-
Then match <status> contains responseStatus
6262
Examples:
6363
| agent | result | method | type | container | resource | status |
6464
| Bob | cannot | PUT | rdf | no | R | [403] |
@@ -88,8 +88,8 @@ Feature: Only authenticated agents can read (and only that) a resource when gran
8888
And headers utils.authHeaders(method, testResource.url, agent)
8989
And header Content-Type = 'text/n3'
9090
And request '@prefix solid: <http://www.w3.org/ns/solid/terms#>. _:insert a solid:InsertDeletePatch; solid:inserts { <> a <http://example.org#Foo> . }.'
91+
And retry until responseStatus == <status>
9192
When method <method>
92-
Then status <status>
9393
Examples:
9494
| agent | result | method | type | container | resource | status |
9595
| Bob | cannot | PATCH | rdf | no | R | 403 |
@@ -109,8 +109,8 @@ Feature: Only authenticated agents can read (and only that) a resource when gran
109109
And headers utils.authHeaders(method, testResource.url, agent)
110110
And header Content-Type = 'text/plain'
111111
And request "Bob's text"
112+
And retry until utils.includesExpectedStatus(responseStatus, <status>)
112113
When method <method>
113-
Then match <status> contains responseStatus
114114
Examples:
115115
| agent | result | method | type | container | resource | status |
116116
| Bob | cannot | PUT | plain | no | R | [403] |
@@ -136,8 +136,8 @@ Feature: Only authenticated agents can read (and only that) a resource when gran
136136
* def testResource = utils.testResources[utils.getResourceKey(container, resource, type)]
137137
Given url testResource.url
138138
And headers utils.authHeaders(method, testResource.url, agent)
139+
And retry until utils.includesExpectedStatus(responseStatus, <status>)
139140
When method <method>
140-
Then match <status> contains responseStatus
141141
Examples:
142142
| agent | result | method | type | container | resource | status |
143143
| Bob | cannot | DELETE | plain | no | R | [403] |

‎web-access-control/protected-operation/read-access-bob.feature‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -18,8 +18,8 @@ Feature: Only Bob can read (and only that) a resource when granted read access
1818
* def testResource = utils.testResources[utils.getResourceKey(container, resource, type)]
1919
Given url testResource.url
2020
And headers utils.authHeaders(method, testResource.url, agent)
21+
And retry until responseStatus == <status>
2122
When method <method>
22-
Then status <status>
2323
Examples:
2424
| agent | result | method | type | container | resource | status |
2525
| Bob | can | GET | plain | no | R | 200 |
@@ -57,8 +57,8 @@ Feature: Only Bob can read (and only that) a resource when granted read access
5757
And headers utils.authHeaders(method, testResource.url, agent)
5858
And header Content-Type = 'text/turtle'
5959
And request '@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#>. <> rdfs:comment "Bob added this.".'
60+
And retry until utils.includesExpectedStatus(responseStatus, <status>)
6061
When method <method>
61-
Then match <status> contains responseStatus
6262
Examples:
6363
| agent | result | method | type | container | resource | status |
6464
| Bob | cannot | PUT | rdf | no | R | [403] |
@@ -88,8 +88,8 @@ Feature: Only Bob can read (and only that) a resource when granted read access
8888
And headers utils.authHeaders(method, testResource.url, agent)
8989
And header Content-Type = 'text/n3'
9090
And request '@prefix solid: <http://www.w3.org/ns/solid/terms#>. _:insert a solid:InsertDeletePatch; solid:inserts { <> a <http://example.org#Foo> . }.'
91+
And retry until responseStatus == <status>
9192
When method <method>
92-
Then status <status>
9393
Examples:
9494
| agent | result | method | type | container | resource | status |
9595
| Bob | cannot | PATCH | rdf | no | R | 403 |
@@ -109,8 +109,8 @@ Feature: Only Bob can read (and only that) a resource when granted read access
109109
And headers utils.authHeaders(method, testResource.url, agent)
110110
And header Content-Type = 'text/plain'
111111
And request "Bob's text"
112+
And retry until utils.includesExpectedStatus(responseStatus, <status>)
112113
When method <method>
113-
Then match <status> contains responseStatus
114114
Examples:
115115
| agent | result | method | type | container | resource | status |
116116
| Bob | cannot | PUT | plain | no | R | [403] |
@@ -136,8 +136,8 @@ Feature: Only Bob can read (and only that) a resource when granted read access
136136
* def testResource = utils.testResources[utils.getResourceKey(container, resource, type)]
137137
Given url testResource.url
138138
And headers utils.authHeaders(method, testResource.url, agent)
139+
And retry until utils.includesExpectedStatus(responseStatus, <status>)
139140
When method <method>
140-
Then match <status> contains responseStatus
141141
Examples:
142142
| agent | result | method | type | container | resource | status |
143143
| Bob | cannot | DELETE | plain | no | R | [403] |

‎web-access-control/protected-operation/read-access-public.feature‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -18,8 +18,8 @@ Feature: Public agents can read (and only that) a resource when granted read acc
1818
* def testResource = utils.testResources[utils.getResourceKey(container, resource, type)]
1919
Given url testResource.url
2020
And headers utils.authHeaders(method, testResource.url, agent)
21+
And retry until responseStatus == <status>
2122
When method <method>
22-
Then status <status>
2323
Examples:
2424
| agent | result | method | type | container | resource | status |
2525
| Bob | can | GET | plain | no | R | 200 |
@@ -61,8 +61,8 @@ Feature: Public agents can read (and only that) a resource when granted read acc
6161
And headers utils.authHeaders(method, testResource.url, agent)
6262
And header Content-Type = 'text/turtle'
6363
And request '@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#>. <> rdfs:comment "Bob added this.".'
64+
And retry until utils.includesExpectedStatus(responseStatus, <status>)
6465
When method <method>
65-
Then match <status> contains responseStatus
6666
Examples:
6767
| agent | result | method | type | container | resource | status |
6868
| Bob | cannot | PUT | rdf | no | R | [403] |
@@ -92,8 +92,8 @@ Feature: Public agents can read (and only that) a resource when granted read acc
9292
And headers utils.authHeaders(method, testResource.url, agent)
9393
And header Content-Type = 'text/n3'
9494
And request '@prefix solid: <http://www.w3.org/ns/solid/terms#>. _:insert a solid:InsertDeletePatch; solid:inserts { <> a <http://example.org#Foo> . }.'
95+
And retry until responseStatus == <status>
9596
When method <method>
96-
Then status <status>
9797
Examples:
9898
| agent | result | method | type | container | resource | status |
9999
| Bob | cannot | PATCH | rdf | no | R | 403 |
@@ -113,8 +113,8 @@ Feature: Public agents can read (and only that) a resource when granted read acc
113113
And headers utils.authHeaders(method, testResource.url, agent)
114114
And header Content-Type = 'text/plain'
115115
And request "Bob's text"
116+
And retry until utils.includesExpectedStatus(responseStatus, <status>)
116117
When method <method>
117-
Then match <status> contains responseStatus
118118
Examples:
119119
| agent | result | method | type | container | resource | status |
120120
| Bob | cannot | PUT | plain | no | R | [403] |
@@ -140,8 +140,8 @@ Feature: Public agents can read (and only that) a resource when granted read acc
140140
* def testResource = utils.testResources[utils.getResourceKey(container, resource, type)]
141141
Given url testResource.url
142142
And headers utils.authHeaders(method, testResource.url, agent)
143+
And retry until utils.includesExpectedStatus(responseStatus, <status>)
143144
When method <method>
144-
Then match <status> contains responseStatus
145145
Examples:
146146
| agent | result | method | type | container | resource | status |
147147
| Bob | cannot | DELETE | plain | no | R | [403] |

‎web-access-control/protected-operation/write-access-agent.feature‎

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -18,8 +18,8 @@ Feature: Only authenticated agents can write (and only that) a resource when gra
1818
* def testResource = utils.testResources[utils.getResourceKey(container, resource, type)]
1919
Given url testResource.url
2020
And headers utils.authHeaders(method, testResource.url, agent)
21+
And retry until responseStatus == <status>
2122
When method <method>
22-
Then status <status>
2323
Examples:
2424
| agent | result | method | type | container | resource | status |
2525
| Bob | cannot | GET | plain | no | WAC | 403 |
@@ -58,15 +58,15 @@ Feature: Only authenticated agents can write (and only that) a resource when gra
5858
And headers utils.authHeaders(method, testResource.url, agent)
5959
And header Content-Type = requestData.contentType
6060
And request requestData.requestBody
61+
And retry until utils.includesExpectedStatus(responseStatus, <writeStatus>)
6162
When method <method>
62-
Then match <writeStatus> contains responseStatus
6363
# Server may return payload with information about the operation e.g. "Created" so check it hasn't leaked the data which was PUT
6464
And string responseString = response
6565
And match responseString !contains requestData.responseShouldNotContain
6666

6767
Given headers utils.authHeaders('GET', testResource.url, agent)
68+
And retry until responseStatus == <readStatus>
6869
When method GET
69-
Then status <readStatus>
7070

7171
Examples:
7272
| agent | result | method | type | container | resource | writeStatus | readStatus |
@@ -94,15 +94,15 @@ Feature: Only authenticated agents can write (and only that) a resource when gra
9494
And headers utils.authHeaders(method, testResource.url, agent)
9595
And header Content-Type = requestData.contentType
9696
And request requestData.requestBody
97+
And retry until utils.includesExpectedStatus(responseStatus, <writeStatus>)
9798
When method <method>
98-
Then match <writeStatus> contains responseStatus
9999
# Server may return payload with information about the operation e.g. "Created" so check it hasn't leaked the data which was PUT
100100
And string responseString = response
101101
And match responseString !contains requestData.responseShouldNotContain
102102

103103
Given headers utils.authHeaders('GET', testResource.url, agent)
104+
And retry until responseStatus == <readStatus>
104105
When method GET
105-
Then status <readStatus>
106106

107107
Examples:
108108
| agent | result | method | type | container | resource | writeStatus | readStatus |
@@ -120,11 +120,12 @@ Feature: Only authenticated agents can write (and only that) a resource when gra
120120
| Public | cannot | PATCH | fictive | WAC | inherited | [401] | 401 |
121121

122122
Scenario Outline: <agent> <result> <method> a <type> resource, when an authenticated agent has <container> access to the container and <resource> access to the resource
123+
123124
* def testResource = utils.createResource(container, resource, type, 'authenticated')
124125
Given url testResource.url
125126
And headers utils.authHeaders(method, testResource.url, agent)
127+
And retry until utils.includesExpectedStatus(responseStatus, <status>)
126128
When method <method>
127-
Then match <status> contains responseStatus
128129
Examples:
129130
| agent | result | method | type | container | resource | status |
130131
| Bob | cannot | DELETE | plain | no | C | [403] |

‎web-access-control/protected-operation/write-access-bob.feature‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -18,8 +18,8 @@ Feature: Only Bob can write (and only that) a resource when granted write access
1818
* def testResource = utils.testResources[utils.getResourceKey(container, resource, type)]
1919
Given url testResource.url
2020
And headers utils.authHeaders(method, testResource.url, agent)
21+
And retry until responseStatus == <status>
2122
When method <method>
22-
Then status <status>
2323
Examples:
2424
| agent | result | method | type | container | resource | status |
2525
| Bob | cannot | GET | plain | no | WAC | 403 |
@@ -58,15 +58,15 @@ Feature: Only Bob can write (and only that) a resource when granted write access
5858
And headers utils.authHeaders(method, testResource.url, agent)
5959
And header Content-Type = requestData.contentType
6060
And request requestData.requestBody
61+
And retry until utils.includesExpectedStatus(responseStatus, <writeStatus>)
6162
When method <method>
62-
Then match <writeStatus> contains responseStatus
6363
# Server may return payload with information about the operation e.g. "Created" so check it hasn't leaked the data which was PUT
6464
And string responseString = response
6565
And match responseString !contains requestData.responseShouldNotContain
6666

6767
Given headers utils.authHeaders('GET', testResource.url, agent)
68+
And retry until responseStatus == <readStatus>
6869
When method GET
69-
Then status <readStatus>
7070

7171
Examples:
7272
| agent | result | method | type | container | resource | writeStatus | readStatus |
@@ -94,15 +94,15 @@ Feature: Only Bob can write (and only that) a resource when granted write access
9494
And headers utils.authHeaders(method, testResource.url, agent)
9595
And header Content-Type = requestData.contentType
9696
And request requestData.requestBody
97+
And retry until utils.includesExpectedStatus(responseStatus, <writeStatus>)
9798
When method <method>
98-
Then match <writeStatus> contains responseStatus
9999
# Server may return payload with information about the operation e.g. "Created" so check it hasn't leaked the data which was PUT
100100
And string responseString = response
101101
And match responseString !contains requestData.responseShouldNotContain
102102

103103
Given headers utils.authHeaders('GET', testResource.url, agent)
104+
And retry until responseStatus == <readStatus>
104105
When method GET
105-
Then status <readStatus>
106106

107107
Examples:
108108
| agent | result | method | type | container | resource | writeStatus | readStatus |
@@ -123,8 +123,8 @@ Feature: Only Bob can write (and only that) a resource when granted write access
123123
* def testResource = utils.createResource(container, resource, type, 'agent', webIds.bob)
124124
Given url testResource.url
125125
And headers utils.authHeaders(method, testResource.url, agent)
126+
And retry until utils.includesExpectedStatus(responseStatus, <status>)
126127
When method <method>
127-
Then match <status> contains responseStatus
128128
Examples:
129129
| agent | result | method | type | container | resource | status |
130130
| Bob | cannot | DELETE | plain | no | C | [403] |

‎web-access-control/protected-operation/write-access-public.feature‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -17,8 +17,8 @@ Feature: Only authenticated agents can write (and only that) a resource when gra
1717
Scenario Outline: <agent> <result> read a <type> resource (<method>), when a public agent has <container> access to the container and <resource> access to the resource
1818
* def testResource = utils.testResources[utils.getResourceKey(container, resource, type)]
1919
Given url testResource.url
20+
And retry until responseStatus == <status>
2021
When method <method>
21-
Then status <status>
2222
Examples:
2323
| agent | result | method | type | container | resource | status |
2424
| Public | cannot | GET | plain | no | WAC | 401 |
@@ -43,8 +43,8 @@ Feature: Only authenticated agents can write (and only that) a resource when gra
4343
Given url testResource.url
4444
And header Content-Type = requestData.contentType
4545
And request requestData.requestBody
46+
And retry until utils.includesExpectedStatus(responseStatus, <writeStatus>)
4647
When method <method>
47-
Then match <writeStatus> contains responseStatus
4848
# Server may return payload with information about the operation e.g. "Created" so check it hasn't leaked the data which was PUT
4949
And string responseString = response
5050
And match responseString !contains requestData.responseShouldNotContain
@@ -71,8 +71,8 @@ Feature: Only authenticated agents can write (and only that) a resource when gra
7171
Given url testResource.url
7272
And header Content-Type = requestData.contentType
7373
And request requestData.requestBody
74+
And retry until utils.includesExpectedStatus(responseStatus, <writeStatus>)
7475
When method <method>
75-
Then match <writeStatus> contains responseStatus
7676
# Server may return payload with information about the operation e.g. "Created" so check it hasn't leaked the data which was PUT
7777
And string responseString = response
7878
And match responseString !contains requestData.responseShouldNotContain
@@ -96,8 +96,8 @@ Feature: Only authenticated agents can write (and only that) a resource when gra
9696
Scenario Outline: <agent> <result> <method> a <type> resource, when a public agent has <container> access to the container and <resource> access to the resource
9797
* def testResource = utils.createResource(container, resource, type, 'public')
9898
Given url testResource.url
99+
And retry until utils.includesExpectedStatus(responseStatus, <status>)
99100
When method <method>
100-
Then match <status> contains responseStatus
101101
Examples:
102102
| agent | result | method | type | container | resource | status |
103103
| Public | cannot | DELETE | plain | no | C | [401] |

0 commit comments

Comments
 (0)