Skip to content

Commit def8ed2

Browse files
Ishakaushik-cloudIshakaushik-cloud
committed
e2e Tests for cloudSQL-MySQL
1 parent 765a7f1 commit def8ed2

12 files changed

Lines changed: 1125 additions & 0 deletions

‎cloudsql-mysql-plugin/src/e2e-test/features/sink/CloudMySqlDesignTime.feature‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
@CloudMySql
2+
Feature: CloudMySql sink- Verify CloudMySql sink plugin design time scenarios
3+
4+
Scenario: To verify CloudMySql sink plugin validation with mandatory properties
5+
Given Open Datafusion Project to configure pipeline
6+
When Expand Plugin group in the LHS plugins list: "Sink"
7+
When Select plugin: "CloudSQL MySQL" from the plugins list as: "Sink"
8+
Then Navigate to the properties page of plugin: "CloudSQL MySQL"
9+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "cloudsql-mysql"
10+
Then Select radio button plugin property: "instanceType" with value: "public"
11+
Then Enter input plugin property: "connectionName" with value: "cdf-athena:us-central1:sql-automation-test-instance"
12+
Then Enter input plugin property: "referenceName" with value: "RefName"
13+
Then Enter input plugin property: "database" with value: "TestDatabase"
14+
Then Enter input plugin property: "tableName" with value: "mytable"
15+
Then Validate "CloudSQL MySQL" plugin properties
16+
Then Close the Plugin Properties page
17+
18+
Scenario: To verify CloudMySql sink plugin validation with connection and basic details for connectivity
19+
Given Open Datafusion Project to configure pipeline
20+
When Expand Plugin group in the LHS plugins list: "Sink"
21+
When Select plugin: "CloudSQL MySQL" from the plugins list as: "Sink"
22+
Then Navigate to the properties page of plugin: "CloudSQL MySQL"
23+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "cloudsql-mysql"
24+
Then Select radio button plugin property: "instanceType" with value: "public"
25+
Then Enter input plugin property: "connectionName" with value: "cdf-athena:us-central1:sql-automation-test-instance"
26+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
27+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
28+
Then Enter key value pairs for plugin property: "connectionArguments" with values from json: "connectionArgumentsList"
29+
Then Enter input plugin property: "referenceName" with value: "RefName"
30+
Then Enter input plugin property: "database" with value: "TestDatabase"
31+
Then Enter input plugin property: "tableName" with value: "mytable"
32+
Then Validate "CloudSQL MySQL" plugin properties
33+
Then Close the Plugin Properties page
34+
35+
36+

‎cloudsql-mysql-plugin/src/e2e-test/features/sink/CloudMySqlDesignTimeValidation.feature‎

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
Feature: CloudMySql sink- Verify ,Mysql sink plugin design time validation scenarios
2+
3+
Scenario: To verify CloudMySql sink plugin validation error message with invalid database
4+
Given Open Datafusion Project to configure pipeline
5+
When Expand Plugin group in the LHS plugins list: "Source"
6+
When Select plugin: "CloudSQL MySQL" from the plugins list as: "Source"
7+
When Expand Plugin group in the LHS plugins list: "Sink"
8+
When Select plugin: "CloudSQL MySQL" from the plugins list as: "Sink"
9+
Then Connect plugins: "CloudSQL MySQL" and "CloudSQL MySQL2" to establish connection
10+
Then Navigate to the properties page of plugin: "CloudSQL MySQL"
11+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "cloudsql-mysql"
12+
Then Select radio button plugin property: "instanceType" with value: "public"
13+
Then Enter input plugin property: "connectionName" with value: "cdf-athena:us-central1:sql-automation-test-instance"
14+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
15+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
16+
Then Enter input plugin property: "referenceName" with value: "RefName"
17+
Then Enter input plugin property: "database" with value: "TestDatabase"
18+
Then Enter textarea plugin property: "importQuery" with value: "insertQuery"
19+
# Then Click on the Get Schema button
20+
# Then Verify the Output Schema matches the Expected Schema: "outputSchema"
21+
# Then Validate "CloudSQL MySQL" plugin properties
22+
Then Close the Plugin Properties page
23+
Then Navigate to the properties page of plugin: "CloudSQL MySQL2"
24+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "cloudsql-mysql"
25+
Then Select radio button plugin property: "instanceType" with value: "public"
26+
Then Enter input plugin property: "connectionName" with value: "cdf-athena:us-central1:sql-automation-test-instance"
27+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
28+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
29+
Then Enter input plugin property: "referenceName" with value: "RefName"
30+
Then Enter input plugin property: "database" with value: "invalidDatabaseName"
31+
Then Enter input plugin property: "tableName" with value: "mytable"
32+
Then Click on the Validate button
33+
# Then Verify that the Plugin is displaying an error message: "errorMessageInvalidSinkDatabase" on the header
34+
35+
Scenario: To verify CloudMySql sink plugin validation error message with invalid tablename
36+
Given Open Datafusion Project to configure pipeline
37+
When Expand Plugin group in the LHS plugins list: "Source"
38+
When Select plugin: "CloudSQL MySQL" from the plugins list as: "Source"
39+
When Expand Plugin group in the LHS plugins list: "Sink"
40+
When Select plugin: "CloudSQL MySQL" from the plugins list as: "Sink"
41+
Then Connect plugins: "CloudSQL MySQL" and "CloudSQL MySQL2" to establish connection
42+
Then Navigate to the properties page of plugin: "CloudSQL MySQL"
43+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "cloudsql-mysql"
44+
Then Select radio button plugin property: "instanceType" with value: "public"
45+
Then Enter input plugin property: "connectionName" with value: "cdf-athena:us-central1:sql-automation-test-instance"
46+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
47+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
48+
Then Enter input plugin property: "referenceName" with value: "RefName"
49+
Then Enter input plugin property: "database" with value: "TestDatabase"
50+
Then Enter textarea plugin property: "importQuery" with value: "insertQuery"
51+
# Then Click on the Get Schema button
52+
# Then Verify the Output Schema matches the Expected Schema: "outputSchema"
53+
# Then Validate "CloudSQL MySQL" plugin properties
54+
Then Close the Plugin Properties page
55+
Then Navigate to the properties page of plugin: "CloudSQL MySQL2"
56+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "cloudsql-mysql"
57+
Then Select radio button plugin property: "instanceType" with value: "public"
58+
Then Enter input plugin property: "connectionName" with value: "cdf-athena:us-central1:sql-automation-test-instance"
59+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
60+
Then Enter input plugin property: "referenceName" with value: "invalidRefName"
61+
Then Enter input plugin property: "database" with value: "TestDatabase"
62+
Then Enter input plugin property: "tableName" with value: "Invalidtable"
63+
Then Click on the Validate button
64+
# Then Verify that the Plugin Property: "table" is displaying an in-line error message: "errorMessageInvalidTableName"
65+
66+
Scenario: To verify CloudMySql sink plugin validation error message with invalid reference Name
67+
Given Open Datafusion Project to configure pipeline
68+
When Expand Plugin group in the LHS plugins list: "Sink"
69+
When Select plugin: "CloudSQL MySQL" from the plugins list as: "Sink"
70+
Then Navigate to the properties page of plugin: "CloudSQL MySQL"
71+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "cloudsql-mysql"
72+
Then Select radio button plugin property: "instanceType" with value: "public"
73+
Then Enter input plugin property: "connectionName" with value: "cdf-athena:us-central1:sql-automation-test-instance"
74+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
75+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
76+
Then Enter input plugin property: "referenceName" with value: "invalidRef"
77+
Then Enter key value pairs for plugin property: "connectionArguments" with values from json: "connectionArgumentsList"
78+
Then Enter input plugin property: "database" with value: "TestDatabase"
79+
Then Enter input plugin property: "tableName" with value: "mytable"
80+
Then Click on the Validate button
81+
Then Verify that the Plugin Property: "referenceName" is displaying an in-line error message: "errorMessageCloudMySqlInvalidReferenceName"
82+
83+
Scenario: To verify CloudMySql sink plugin validation error message with blank username
84+
Given Open Datafusion Project to configure pipeline
85+
When Expand Plugin group in the LHS plugins list: "Sink"
86+
When Select plugin: "CloudSQL MySQL" from the plugins list as: "Sink"
87+
Then Navigate to the properties page of plugin: "CloudSQL MySQL"
88+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "cloudsql-mysql"
89+
Then Select radio button plugin property: "instanceType" with value: "public"
90+
Then Enter input plugin property: "connectionName" with value: "cdf-athena:us-central1:sql-automation-test-instance"
91+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
92+
Then Enter key value pairs for plugin property: "connectionArguments" with values from json: "connectionArgumentsList"
93+
Then Enter input plugin property: "referenceName" with value: "RefName"
94+
Then Enter input plugin property: "database" with value: "TestDatabase"
95+
Then Enter input plugin property: "tableName" with value: "mytable"
96+
Then Click on the Validate button
97+
Then Verify that the Plugin Property: "user" is displaying an in-line error message: "errorMessageBlankUsername"
98+
99+
Scenario: To verify CloudMySql sink plugin validation error message with ConnectionName
100+
Given Open Datafusion Project to configure pipeline
101+
When Expand Plugin group in the LHS plugins list: "Sink"
102+
When Select plugin: "CloudSQL MySQL" from the plugins list as: "Sink"
103+
Then Navigate to the properties page of plugin: "CloudSQL MySQL"
104+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "cloudsql-mysql"
105+
Then Select radio button plugin property: "instanceType" with value: "public"
106+
Then Enter input plugin property: "connectionName" with value: "cdf-athena:us-central1"
107+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
108+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
109+
Then Enter key value pairs for plugin property: "connectionArguments" with values from json: "connectionArgumentsList"
110+
Then Enter input plugin property: "referenceName" with value: "RefName"
111+
Then Enter input plugin property: "database" with value: "TestDatabase"
112+
Then Enter input plugin property: "tableName" with value: "mytable"
113+
Then Click on the Validate button
114+
Then Verify that the Plugin Property: "connectionName" is displaying an in-line error message: "errorMessageConnectionName"

‎cloudsql-mysql-plugin/src/e2e-test/features/sink/CloudMySqlDesignTimeWithMacro.feature‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
Feature: CloudMySql sink- Verify CloudMySql sink plugin design time macro scenarios
2+
3+
Scenario: To verify CloudMySql sink plugin validation with macro enabled fields for connection section
4+
Given Open Datafusion Project to configure pipeline
5+
When Expand Plugin group in the LHS plugins list: "Sink"
6+
When Select plugin: "CloudSQL MySQL" from the plugins list as: "Sink"
7+
Then Navigate to the properties page of plugin: "CloudSQL MySQL"
8+
Then Click on the Macro button of Property: "jdbcPluginName" and set the value to: "cloudsql-mysql"
9+
Then Enter input plugin property: "connectionName" with value: "cdf-athena:us-central1:sql-automation-test-instance"
10+
Then Select radio button plugin property: "instanceType" with value: "public"
11+
Then Click on the Macro button of Property: "user" and set the value to: "username"
12+
Then Click on the Macro button of Property: "password" and set the value to: "password"
13+
Then Click on the Macro button of Property: "connectionArguments" and set the value to: "connectionArgumentsList"
14+
Then Enter input plugin property: "referenceName" with value: "sourceRef"
15+
Then Replace input plugin property: "database" with value: "TestDatabase"
16+
Then Click on the Validate button
17+
# Then Validate "CloudSQL MySQL" plugin properties
18+
Then Close the Plugin Properties page
19+
20+
Scenario: To verify cloudsql sink plugin validation with macro enabled fields for basic section
21+
Given Open Datafusion Project to configure pipeline
22+
When Expand Plugin group in the LHS plugins list: "Sink"
23+
When Select plugin: "CloudSQL MySQL" from the plugins list as: "Sink"
24+
Then Navigate to the properties page of plugin: "CloudSQL MySQL"
25+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "cloudsql-mysql"
26+
Then Enter input plugin property: "connectionName" with value: "cdf-athena:us-central1:sql-automation-test-instance"
27+
Then Select radio button plugin property: "instanceType" with value: "public"
28+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
29+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
30+
Then Enter input plugin property: "referenceName" with value: "sourceRef"
31+
Then Replace input plugin property: "database" with value: "TestDatabase"
32+
Then Click on the Macro button of Property: "tableName" and set the value to: "mytable"
33+
Then Click on the Validate button
34+
# Then Validate "CloudSQL MySQL" plugin properties
35+
Then Close the Plugin Properties page

‎cloudsql-mysql-plugin/src/e2e-test/features/sink/CloudMySqlRunTime.feature‎

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
Feature: CloudMySql - Verify data transfer from BigQuery source to CloudMySql sink
2+
3+
Scenario: To verify data is getting transferred from BigQuery source to CloudMySql sink successfully
4+
Given Open Datafusion Project to configure pipeline
5+
When Expand Plugin group in the LHS plugins list: "Source"
6+
When Select plugin: "BigQuery" from the plugins list as: "Source"
7+
When Expand Plugin group in the LHS plugins list: "Sink"
8+
When Select plugin: "CloudSQL MySQL" from the plugins list as: "Sink"
9+
Then Connect plugins: "BigQuery" and "CloudSQL MySQL" to establish connection
10+
Then Navigate to the properties page of plugin: "BigQuery"
11+
Then Replace input plugin property: "project" with value: "projectId"
12+
Then Enter input plugin property: "datasetProject" with value: "projectId"
13+
Then Override Service account details if set in environment variables
14+
Then Enter input plugin property: "referenceName" with value: "BQReferenceName"
15+
Then Enter input plugin property: "dataset" with value: "dataset"
16+
Then Enter input plugin property: "table" with value: "bqTargetTable"
17+
# Then Click on the Get Schema button
18+
# Then Validate "BigQuery" plugin properties
19+
Then Close the Plugin Properties page
20+
Then Navigate to the properties page of plugin: "CloudSQL MySQL"
21+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "cloudsql-mysql"
22+
Then Select radio button plugin property: "instanceType" with value: "public"
23+
Then Enter input plugin property: "connectionName" with value: "cdf-athena:us-central1:sql-automation-test-instance"
24+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
25+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
26+
Then Enter input plugin property: "referenceName" with value: "RefName"
27+
Then Enter input plugin property: "database" with value: "TestDatabase"
28+
Then Enter input plugin property: "tableName" with value: "mytable"
29+
# Then Validate "CloudSQL MySQL" plugin properties
30+
Then Close the Plugin Properties page
31+
Then Save the pipeline
32+
Then Preview and run the pipeline
33+
Then Wait till pipeline preview is in running state
34+
Then Open and capture pipeline preview logs
35+
# Then Verify the preview run status of pipeline in the logs is "succeeded"
36+
# Then Close the pipeline logs
37+
# Then Close the preview
38+
# Then Deploy the pipeline
39+
# Then Run the Pipeline in Runtime
40+
# Then Wait till pipeline is in running state
41+
# Then Open and capture logs
42+
# Then Verify the pipeline status is "Succeeded"
43+
# Then Close the pipeline logs
44+
# Then Validate OUT record count is equal to records transferred to target BigQuery table
45+
46+
Scenario: To verify data is getting transferred from BigQuery source to CloudMySql sink successfully when connection arguments are set
47+
Given Open Datafusion Project to configure pipeline
48+
When Expand Plugin group in the LHS plugins list: "Source"
49+
When Select plugin: "BigQuery" from the plugins list as: "Source"
50+
When Expand Plugin group in the LHS plugins list: "Sink"
51+
When Select plugin: "CloudSQL MySQL" from the plugins list as: "Sink"
52+
Then Connect plugins: "BigQuery" and "CloudSQL MySQL" to establish connection
53+
Then Navigate to the properties page of plugin: "BigQuery"
54+
Then Replace input plugin property: "project" with value: "projectId"
55+
Then Enter input plugin property: "datasetProject" with value: "projectId"
56+
Then Override Service account details if set in environment variables
57+
Then Enter input plugin property: "referenceName" with value: "BQReferenceName"
58+
Then Enter input plugin property: "dataset" with value: "dataset"
59+
Then Enter input plugin property: "table" with value: "bqTargetTable"
60+
# Then Click on the Get Schema button
61+
# Then Validate "BigQuery" plugin properties
62+
Then Close the Plugin Properties page
63+
Then Navigate to the properties page of plugin: "CloudSQL MySQL"
64+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "cloudsql-mysql"
65+
Then Select radio button plugin property: "instanceType" with value: "public"
66+
Then Enter input plugin property: "connectionName" with value: "cdf-athena:us-central1:sql-automation-test-instance"
67+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
68+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
69+
Then Enter input plugin property: "referenceName" with value: "RefName"
70+
Then Enter input plugin property: "database" with value: "TestDatabase"
71+
Then Enter input plugin property: "tableName" with value: "mytable"
72+
Then Enter key value pairs for plugin property: "connectionArguments" with values from json: "connectionArgumentsList"
73+
# Then Validate "CloudSQL MySQL" plugin properties
74+
Then Close the Plugin Properties page
75+
Then Save the pipeline
76+
Then Preview and run the pipeline
77+
Then Wait till pipeline preview is in running state
78+
# Then Open and capture pipeline preview logs
79+
# Then Verify the preview run status of pipeline in the logs is "succeeded"
80+
# Then Close the pipeline logs
81+
# Then Close the preview
82+
# Then Deploy the pipeline
83+
# Then Run the Pipeline in Runtime
84+
# Then Wait till pipeline is in running state
85+
# Then Open and capture logs
86+
# Then Verify the pipeline status is "Succeeded"
87+
# Then Close the pipeline logs
88+
# Then Validate OUT record count is equal to records transferred to target BigQuery table
89+
90+
91+
92+

0 commit comments

Comments
 (0)