Skip to content

Commit e56ccf1

Browse files
edwardsphedwardsph
authored
Make CORS test detect empty responses better (#117)
* Set the origin in cors tests from config so that it uses the same one. * Cast response to string when checking HEAD requests in CORS. * Update test subjects
1 parent a430fc9 commit e56ccf1

9 files changed

Lines changed: 49 additions & 40 deletions

‎CHANGELOG.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,10 @@
11
# Solid Specification Conformance Tests
22

3+
## Release 0.0.19
4+
* Set the origin in CORS tests from config so that it uses the same one.
5+
* Cast response to string when checking HEAD requests in CORS.
6+
* Add retries to authorization tests to allow for servers which propagate access policies asynchronously.
7+
38
## Release 0.0.18
49
* Update run.sh to support CSS 7.
510
* Resolve location headers against base URL before using them in case they are relative.

‎protocol/cors/acao-vary.feature‎

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -7,10 +7,11 @@ Feature: Server returns correct Access-Control-Allow-Origin and Vary headers
77
Scenario Outline: Access-Control-Allow-Origin header is set to correct origin for <method> on container
88
Given url testContainer.url
99
And headers clients.alice.getAuthHeaders('<method>', testContainer.url)
10-
And header Origin = 'https://tester'
10+
And header Origin = config.origin
1111
When method <method>
1212
Then match <statuses> contains responseStatus
13-
And match header Access-Control-Allow-Origin == 'https://tester'
13+
And match header Access-Control-Allow-Origin == config.origin
14+
* string response = response
1415
And match response <check>
1516
Examples:
1617
| method | statuses | check |
@@ -21,10 +22,11 @@ Feature: Server returns correct Access-Control-Allow-Origin and Vary headers
2122
Scenario Outline: Vary header includes Origin for <method> on container
2223
Given url testContainer.url
2324
And headers clients.alice.getAuthHeaders('<method>', testContainer.url)
24-
And header Origin = 'https://tester'
25+
And header Origin = config.origin
2526
When method <method>
2627
Then match <statuses> contains responseStatus
2728
And match header Vary contains 'Origin'
29+
* string response = response
2830
And match response <check>
2931
Examples:
3032
| method | statuses | check |
@@ -35,10 +37,11 @@ Feature: Server returns correct Access-Control-Allow-Origin and Vary headers
3537
Scenario Outline: Access-Control-Allow-Origin header is set to correct origin for <method> on resource
3638
Given url resource.url
3739
And headers clients.alice.getAuthHeaders('<method>', resource.url)
38-
And header Origin = 'https://tester'
40+
And header Origin = config.origin
3941
When method <method>
4042
Then match <statuses> contains responseStatus
41-
And match header Access-Control-Allow-Origin == 'https://tester'
43+
And match header Access-Control-Allow-Origin == config.origin
44+
* string response = response
4245
And match response <check>
4346
Examples:
4447
| method | statuses | check |
@@ -49,10 +52,11 @@ Feature: Server returns correct Access-Control-Allow-Origin and Vary headers
4952
Scenario Outline: Vary header includes Origin for <method> on resource
5053
Given url resource.url
5154
And headers clients.alice.getAuthHeaders('<method>', resource.url)
52-
And header Origin = 'https://tester'
55+
And header Origin = config.origin
5356
When method <method>
5457
Then match <statuses> contains responseStatus
5558
And match header Vary contains 'Origin'
59+
* string response = response
5660
And match response <check>
5761
Examples:
5862
| method | statuses | check |

‎protocol/cors/accept-acah.feature‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ Feature: Server should explicitly list Accept under Access-Control-Allow-Headers
88
Scenario: OPTIONS request doesn't return Accept in Access-Control-Allow-Headers for GET pre-flight if not requested
99
Given url testContainer.url
1010
And headers clients.alice.getAuthHeaders('OPTIONS', testContainer.url)
11-
And header Origin = 'https://tester'
11+
And header Origin = config.origin
1212
And header Access-Control-Request-Method = 'GET'
1313
And header Access-Control-Request-Headers = 'X-CUSTOM, Content-Type'
1414
When method OPTIONS
@@ -19,7 +19,7 @@ Feature: Server should explicitly list Accept under Access-Control-Allow-Headers
1919
Scenario: OPTIONS request returns Accept in Access-Control-Allow-Headers for POST pre-flight
2020
Given url testContainer.url
2121
And headers clients.alice.getAuthHeaders('OPTIONS', testContainer.url)
22-
And header Origin = 'https://tester'
22+
And header Origin = config.origin
2323
And header Access-Control-Request-Method = 'POST'
2424
And header Access-Control-Request-Headers = 'X-CUSTOM, Content-Type, Accept'
2525
When method OPTIONS
@@ -30,7 +30,7 @@ Feature: Server should explicitly list Accept under Access-Control-Allow-Headers
3030
Scenario: OPTIONS request returns Accept in Access-Control-Allow-Headers for GET pre-flight with long Accept
3131
Given url testContainer.url
3232
And headers clients.alice.getAuthHeaders('OPTIONS', testContainer.url)
33-
And header Origin = 'https://tester'
33+
And header Origin = config.origin
3434
And header Access-Control-Request-Method = 'GET'
3535
And header Access-Control-Request-Headers = 'X-CUSTOM, Content-Type, Accept'
3636
When method OPTIONS

‎protocol/cors/access-control-headers.feature‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -9,12 +9,12 @@ Feature: Server must respond to requests sending Origin with the appropriate Acc
99

1010
Scenario Outline: Simple request: <method> request returns access control headers
1111
Given url testContainer.url
12-
And header Origin = 'https://tester'
12+
And header Origin = config.origin
1313
And headers <headers>
1414
* <body>
1515
When method <method>
1616
Then match <statuses> contains responseStatus
17-
And match header Access-Control-Allow-Origin == 'https://tester'
17+
And match header Access-Control-Allow-Origin == config.origin
1818
Examples:
1919
| method | headers! | body | statuses |
2020
| GET | {Accept: 'text/turtle'} | def ignore = 1 | [401] |
@@ -24,12 +24,12 @@ Feature: Server must respond to requests sending Origin with the appropriate Acc
2424
Scenario Outline: Requests with credentials: <method> request returns access control headers
2525
Given url testContainer.url
2626
And headers clients.alice.getAuthHeaders('<method>', testContainer.url)
27-
And header Origin = 'https://tester'
27+
And header Origin = config.origin
2828
And headers <headers>
2929
* <body>
3030
When method <method>
3131
Then match <statuses> contains responseStatus
32-
And match header Access-Control-Allow-Origin == 'https://tester'
32+
And match header Access-Control-Allow-Origin == config.origin
3333
Examples:
3434
| method | headers! | body | statuses |
3535
| GET | {Accept: 'text/turtle'} | def ignore = 1 | [200] |

‎protocol/cors/enumerate-headers.feature‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ Feature: Server should enumerate headers in Access-Control-Expose-Headers
77
Scenario: Access-Control-Expose-Headers is present but not *
88
Given url testContainer.url
99
And headers clients.alice.getAuthHeaders('GET', testContainer.url)
10-
And header Origin = 'https://tester'
10+
And header Origin = config.origin
1111
And header Accept = 'text/turtle'
1212
When method GET
1313
Then status 200

‎protocol/cors/preflight-requests.feature‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -6,12 +6,12 @@ Feature: Server must implement the CORS protocol for preflight requests
66

77
Scenario Outline: Pre-flight CORS request for <method> request
88
Given url testContainer.url
9-
And header Origin = 'https://tester'
9+
And header Origin = config.origin
1010
And header Access-Control-Request-Method = '<method>'
1111
And header Access-Control-Request-Headers = 'X-CUSTOM, Content-Type, Accept'
1212
When method OPTIONS
1313
Then match [200, 204] contains responseStatus
14-
And match header Access-Control-Allow-Origin == 'https://tester'
14+
And match header Access-Control-Allow-Origin == config.origin
1515
And match header Access-Control-Allow-Methods contains '<method>'
1616
And match header Access-Control-Allow-Headers contains 'X-CUSTOM'
1717
And match header Access-Control-Allow-Headers contains 'Content-Type'
@@ -20,14 +20,14 @@ Feature: Server must implement the CORS protocol for preflight requests
2020
And match response == ''
2121

2222
Given url testContainer.url
23-
And header Origin = 'https://tester'
23+
And header Origin = config.origin
2424
And headers clients.alice.getAuthHeaders('<method>', testContainer.url)
2525
# Demonstrates the case where a long Accept header is allowed
2626
And header Accept = 'text/turtle;q=0.9, application/rdf+xml;q=0.8, application/n-triples;q=0.8, application/n-quads;q=0.8, text/x-nquads;q=0.8, application/trig;q=0.8, text/n3;q=0.8, application/ld+json;q=0.8, application/x-binary-rdf;q=0.8, text/plain;q=0.7'
2727
* <body>
2828
When method <method>
2929
Then match <statuses> contains responseStatus
30-
And match header Access-Control-Allow-Origin == 'https://tester'
30+
And match header Access-Control-Allow-Origin == config.origin
3131
And match header Access-Control-Expose-Headers != null
3232
And match header Access-Control-Expose-Headers != '*'
3333
# Check Content-Type on GET request only
@@ -44,20 +44,20 @@ Feature: Server must implement the CORS protocol for preflight requests
4444
Scenario: OPTIONS request returns headers for pre-flight check after redirect from http
4545
* configure followRedirects = false
4646
Given url testContainer.url.replace(/^https:/, 'http:')
47-
And header Origin = 'https://tester'
47+
And header Origin = config.origin
4848
And header Access-Control-Request-Method = 'POST'
4949
And header Access-Control-Request-Headers = 'X-CUSTOM, Content-Type'
5050
When method OPTIONS
5151
Then match [301, 308] contains responseStatus
5252
* def location = resolveUri(testContainer.url, karate.response.headerValues('location')[0])
5353

5454
Given url location
55-
And header Origin = 'https://tester'
55+
And header Origin = config.origin
5656
And header Access-Control-Request-Method = 'POST'
5757
And header Access-Control-Request-Headers = 'X-CUSTOM, Content-Type'
5858
When method OPTIONS
5959
Then match [200, 204] contains responseStatus
60-
And match header Access-Control-Allow-Origin == 'https://tester'
60+
And match header Access-Control-Allow-Origin == config.origin
6161
And match header Access-Control-Allow-Methods contains 'POST'
6262
And match header Access-Control-Allow-Headers contains 'X-CUSTOM'
6363
And match header Access-Control-Allow-Headers contains 'Content-Type'

‎protocol/cors/preflight.feature‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -5,12 +5,12 @@ Feature: Server must support HTTP OPTIONS for CORS preflight requests
55

66
Scenario: OPTIONS request returns headers for pre-flight check
77
Given url testContainer.url
8-
And header Origin = 'https://tester'
8+
And header Origin = config.origin
99
And header Access-Control-Request-Method = 'POST'
1010
And header Access-Control-Request-Headers = 'X-CUSTOM, Content-Type'
1111
When method OPTIONS
1212
Then match [200, 204] contains responseStatus
13-
And match header Access-Control-Allow-Origin == 'https://tester'
13+
And match header Access-Control-Allow-Origin == config.origin
1414
And match header Access-Control-Allow-Methods contains 'POST'
1515
And match header Access-Control-Allow-Headers contains 'X-CUSTOM'
1616
And match header Access-Control-Allow-Headers contains 'Content-Type'
@@ -21,20 +21,20 @@ Feature: Server must support HTTP OPTIONS for CORS preflight requests
2121
@http-redirect
2222
Scenario: OPTIONS request returns headers for pre-flight check after redirect from http
2323
Given url testContainer.url.replace(/^https:/, 'http:')
24-
And header Origin = 'https://tester'
24+
And header Origin = config.origin
2525
And header Access-Control-Request-Method = 'POST'
2626
And header Access-Control-Request-Headers = 'X-CUSTOM, Content-Type'
2727
When method OPTIONS
2828
Then match [301, 308] contains responseStatus
2929
* def location = resolveUri(testContainer.url, karate.response.headerValues('location')[0])
3030

3131
Given url location
32-
And header Origin = 'https://tester'
32+
And header Origin = config.origin
3333
And header Access-Control-Request-Method = 'POST'
3434
And header Access-Control-Request-Headers = 'X-CUSTOM, Content-Type'
3535
When method OPTIONS
3636
Then match [200, 204] contains responseStatus
37-
And match header Access-Control-Allow-Origin == 'https://tester'
37+
And match header Access-Control-Allow-Origin == config.origin
3838
And match header Access-Control-Allow-Methods contains 'POST'
3939
And match header Access-Control-Allow-Headers contains 'X-CUSTOM'
4040
And match header Access-Control-Allow-Headers contains 'Content-Type'

‎protocol/cors/simple-requests.feature‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -9,12 +9,12 @@ Feature: Server must implement the CORS protocol for simple requests
99

1010
Scenario Outline: Simple container request: <method> request returns access control headers
1111
Given url testContainer.url
12-
And header Origin = 'https://tester'
12+
And header Origin = config.origin
1313
And headers <headers>
1414
* <body>
1515
When method <method>
1616
Then match <statuses> contains responseStatus
17-
And match header Access-Control-Allow-Origin == 'https://tester'
17+
And match header Access-Control-Allow-Origin == config.origin
1818
And match header Access-Control-Expose-Headers != null
1919
And match header Access-Control-Expose-Headers != '*'
2020
Examples:
@@ -25,12 +25,12 @@ Feature: Server must implement the CORS protocol for simple requests
2525

2626
Scenario Outline: Simple resource request: <method> request returns access control headers
2727
Given url resource.url
28-
And header Origin = 'https://tester'
28+
And header Origin = config.origin
2929
And headers <headers>
3030
* <body>
3131
When method <method>
3232
Then match <statuses> contains responseStatus
33-
And match header Access-Control-Allow-Origin == 'https://tester'
33+
And match header Access-Control-Allow-Origin == config.origin
3434
And match header Access-Control-Expose-Headers != null
3535
And match header Access-Control-Expose-Headers != '*'
3636
Examples:
@@ -41,12 +41,12 @@ Feature: Server must implement the CORS protocol for simple requests
4141
Scenario Outline: Requests container with credentials: <method> request returns access control headers
4242
Given url testContainer.url
4343
And headers clients.alice.getAuthHeaders('<method>', testContainer.url)
44-
And header Origin = 'https://tester'
44+
And header Origin = config.origin
4545
And headers <headers>
4646
* <body>
4747
When method <method>
4848
Then match <statuses> contains responseStatus
49-
And match header Access-Control-Allow-Origin == 'https://tester'
49+
And match header Access-Control-Allow-Origin == config.origin
5050
And match header Access-Control-Expose-Headers != null
5151
And match header Access-Control-Expose-Headers != '*'
5252
# Check Vary on GET/HEAD requests only
@@ -60,12 +60,12 @@ Feature: Server must implement the CORS protocol for simple requests
6060
Scenario Outline: Requests resource with credentials: <method> request returns access control headers
6161
Given url resource.url
6262
And headers clients.alice.getAuthHeaders('<method>', resource.url)
63-
And header Origin = 'https://tester'
63+
And header Origin = config.origin
6464
And headers <headers>
6565
* <body>
6666
When method <method>
6767
Then match <statuses> contains responseStatus
68-
And match header Access-Control-Allow-Origin == 'https://tester'
68+
And match header Access-Control-Allow-Origin == config.origin
6969
And match header Access-Control-Expose-Headers != null
7070
And match header Access-Control-Expose-Headers != '*'
7171
And match header Vary contains 'Origin'

‎test-subjects.ttl‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -23,8 +23,8 @@
2323
rdfs:comment "Access for anonymous users is not supported nor required by Solid specifications so tests that depend on this are not run."@en.
2424

2525
<ess#test-subject-release>
26-
doap:revision "2.1";
27-
doap:created "2023-03-28"^^xsd:date .
26+
doap:revision "2.2";
27+
doap:created "2024-03-19"^^xsd:date .
2828

2929
<css>
3030
a earl:Software, earl:TestSubject ;
@@ -37,8 +37,8 @@
3737
solid-test:skip "acp", "http-redirect" .
3838

3939
<css#test-subject-release>
40-
doap:revision "7.0.3" ;
41-
doap:created "2024-01-05"^^xsd:date .
40+
doap:revision "7.0.4" ;
41+
doap:created "2024-02-07"^^xsd:date .
4242

4343
<nss>
4444
a earl:Software, earl:TestSubject ;
@@ -51,8 +51,8 @@
5151
solid-test:skip "acp" .
5252

5353
<nss#test-subject-release>
54-
doap:revision "5.7.7"@en ;
55-
doap:created "2023-03-17"^^xsd:date .
54+
doap:revision "5.7.9"@en ;
55+
doap:created "2024-03-08"^^xsd:date .
5656

5757
<trinpod>
5858
a earl:Software, earl:TestSubject ;

0 commit comments

Comments
 (0)