Skip to content

Commit 67e6e49

Browse files
priyabhatnagar25priyabhatnagar25
committed
CloudSql My Sql Sink Run Time Scenarios e2e tests
1 parent 9fcc646 commit 67e6e49

6 files changed

Lines changed: 156 additions & 107 deletions

File tree

‎cloudsql-mysql-plugin/src/e2e-test/features/sink/CloudMySqlRunTime.feature‎

Lines changed: 0 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -58,49 +58,6 @@ Feature: CloudMySql Sink - Run time scenarios
5858
Then Verify the pipeline status is "Succeeded"
5959
Then Close the pipeline logs
6060

61-
@BQ_SOURCE_TEST @CLOUDMYSQL_SOURCE_TEST
62-
Scenario: To verify data is getting transferred from BigQuery source to CloudMySql sink successfully when connection arguments are set
63-
Given Open Datafusion Project to configure pipeline
64-
When Expand Plugin group in the LHS plugins list: "Source"
65-
When Select plugin: "BigQuery" from the plugins list as: "Source"
66-
When Expand Plugin group in the LHS plugins list: "Sink"
67-
When Select plugin: "CloudSQL MySQL" from the plugins list as: "Sink"
68-
Then Connect plugins: "BigQuery" and "CloudSQL MySQL" to establish connection
69-
Then Navigate to the properties page of plugin: "BigQuery"
70-
Then Replace input plugin property: "project" with value: "projectId"
71-
Then Enter input plugin property: "datasetProject" with value: "projectId"
72-
Then Enter input plugin property: "referenceName" with value: "BQReferenceName"
73-
Then Enter input plugin property: "dataset" with value: "dataset"
74-
Then Enter input plugin property: "table" with value: "bqTargetTable"
75-
Then Click on the Get Schema button
76-
Then Validate "BigQuery" plugin properties
77-
Then Close the Plugin Properties page
78-
Then Navigate to the properties page of plugin: "CloudSQL MySQL"
79-
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "cloudsql-mysql"
80-
Then Select radio button plugin property: "instanceType" with value: "public"
81-
Then Enter input plugin property: "connectionName" with value: "ConnectionName"
82-
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
83-
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
84-
Then Enter input plugin property: "referenceName" with value: "RefName"
85-
Then Enter input plugin property: "database" with value: "TestDatabase"
86-
Then Replace input plugin property: "tableName" with value: "targetTable"
87-
Then Enter key value pairs for plugin property: "connectionArguments" with values from json: "connectionArgumentsList"
88-
Then Validate "CloudSQL MySQL" plugin properties
89-
Then Close the Plugin Properties page
90-
Then Save the pipeline
91-
Then Preview and run the pipeline
92-
Then Wait till pipeline preview is in running state
93-
Then Open and capture pipeline preview logs
94-
Then Verify the preview run status of pipeline in the logs is "succeeded"
95-
Then Close the pipeline logs
96-
Then Close the preview
97-
Then Deploy the pipeline
98-
Then Run the Pipeline in Runtime
99-
Then Wait till pipeline is in running state
100-
Then Open and capture logs
101-
Then Verify the pipeline status is "Succeeded"
102-
Then Close the pipeline logs
103-
10461

10562

10663

‎cloudsql-mysql-plugin/src/e2e-test/features/sink/CloudMySqlRunTimeMacro.feature‎

Lines changed: 132 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@
1515

1616
Feature: CloudMySql Sink - Run time scenarios (macro)
1717

18+
@BQ_SOURCE_TEST @CLOUDMYSQL_TEST_TABLE
1819
Scenario: To verify data is getting transferred from BigQuery source to CloudMySql sink using macro arguments in connection section
1920
Given Open Datafusion Project to configure pipeline
2021
When Expand Plugin group in the LHS plugins list: "Source"
@@ -24,34 +25,28 @@ Feature: CloudMySql Sink - Run time scenarios (macro)
2425
Then Connect plugins: "BigQuery" and "CloudSQL MySQL" to establish connection
2526
Then Navigate to the properties page of plugin: "BigQuery"
2627
Then Enter input plugin property: "referenceName" with value: "BQReferenceName"
27-
Then Click on the Macro button of Property: "project" and set the value to: "projectId"
28-
Then Click on the Macro button of Property: "datasetProject" and set the value to: "bqDatasetProjectId"
29-
Then Click on the Macro button of Property: "dataset" and set the value to: "bqDataset"
30-
Then Click on the Macro button of Property: "table" and set the value to: "bqTargetTable"
31-
Then Click on the Get Schema button
28+
And Replace input plugin property: "project" with value: "projectId"
29+
And Enter input plugin property: "datasetProject" with value: "datasetprojectId"
30+
And Enter input plugin property: "dataset" with value: "dataset"
31+
And Enter input plugin property: "table" with value: "bqSourceTable"
3232
Then Validate "BigQuery" plugin properties
3333
Then Close the Plugin Properties page
3434
Then Navigate to the properties page of plugin: "CloudSQL MySQL"
35-
Then Click on the Macro button of Property: "jdbcPluginName" and set the value to: "cloudsql-mysql"
35+
Then Click on the Macro button of Property: "jdbcPluginName" and set the value to: "driverName"
3636
Then Select radio button plugin property: "instanceType" with value: "public"
3737
Then Enter input plugin property: "connectionName" with value: "ConnectionName"
38-
Then Click on the Macro button of Property: "user" and set the value to: "username"
39-
Then Click on the Macro button of Property: "password" and set the value to: "password"
38+
Then Click on the Macro button of Property: "user" and set the value to: "Username"
39+
Then Click on the Macro button of Property: "password" and set the value to: "Password"
4040
Then Enter input plugin property: "referenceName" with value: "RefName"
41-
Then Enter input plugin property: "database" with value: "TestDatabase"
42-
Then Click on the Macro button of Property: "tableName" and set the value to: "mytable"
41+
Then Enter input plugin property: "database" with value: "DatabaseName"
42+
Then Replace input plugin property: "tableName" with value: "targetTable"
4343
Then Validate "CloudSQL MySQL" plugin properties
4444
Then Close the Plugin Properties page
4545
Then Save the pipeline
4646
Then Preview and run the pipeline
47-
Then Enter runtime argument value "projectId" for key "projectId"
48-
Then Enter runtime argument value "bqDatasetId" for key "bqDatasetProjectId"
49-
Then Enter runtime argument value "dataset" for key "bqDataset"
50-
Then Enter runtime argument value "bqSourceTable" for key "bqTargetTable"
51-
Then Enter runtime argument value "driver" for key "cloudsql-mysql"
52-
Then Enter runtime argument value from environment variable "name" for key "username"
53-
Then Enter runtime argument value from environment variable "pass" for key "password"
54-
Then Enter runtime argument value "table" for key "mytable"
47+
Then Enter runtime argument value "driver" for key "driverName"
48+
Then Enter runtime argument value "username" for key "Username"
49+
Then Enter runtime argument value "password" for key "Password"
5550
Then Run the preview of pipeline with runtime arguments
5651
Then Wait till pipeline preview is in running state
5752
Then Open and capture pipeline preview logs
@@ -60,18 +55,128 @@ Feature: CloudMySql Sink - Run time scenarios (macro)
6055
Then Close the preview
6156
Then Deploy the pipeline
6257
Then Run the Pipeline in Runtime
63-
Then Enter runtime argument value "projectId" for key "projectId"
64-
Then Enter runtime argument value "bqDatasetId" for key "bqDatasetProjectId"
65-
Then Enter runtime argument value "dataset" for key "bqDataset"
66-
Then Enter runtime argument value "bqSourceTable" for key "bqTargetTable"
67-
Then Enter runtime argument value "driver" for key "cloudsql-mysql"
68-
Then Enter runtime argument value from environment variable "name" for key "username"
69-
Then Enter runtime argument value from environment variable "pass" for key "password"
70-
Then Click on the Macro button of Property: "dataset" and set the value to: "bqDataset"
71-
Then Enter runtime argument value "table" for key "mytable"
58+
Then Enter runtime argument value "driver" for key "driverName"
59+
Then Enter runtime argument value "username" for key "Username"
60+
Then Enter runtime argument value "password" for key "Password"
7261
Then Run the Pipeline in Runtime with runtime arguments
7362
Then Wait till pipeline is in running state
7463
Then Open and capture logs
7564
Then Verify the pipeline status is "Succeeded"
7665
Then Close the pipeline logs
7766

67+
@BQ_SOURCE_TEST @CLOUDMYSQL_TEST_TABLE
68+
Scenario: To verify data is getting transferred from BigQuery source to CloudMySql sink using macro arguments in basic section
69+
Given Open Datafusion Project to configure pipeline
70+
When Expand Plugin group in the LHS plugins list: "Source"
71+
When Select plugin: "BigQuery" from the plugins list as: "Source"
72+
When Expand Plugin group in the LHS plugins list: "Sink"
73+
When Select plugin: "CloudSQL MySQL" from the plugins list as: "Sink"
74+
Then Connect plugins: "BigQuery" and "CloudSQL MySQL" to establish connection
75+
Then Navigate to the properties page of plugin: "BigQuery"
76+
And Replace input plugin property: "project" with value: "projectId"
77+
And Enter input plugin property: "datasetProject" with value: "datasetprojectId"
78+
And Enter input plugin property: "dataset" with value: "dataset"
79+
And Enter input plugin property: "table" with value: "bqSourceTable"
80+
Then Validate "BigQuery" plugin properties
81+
Then Close the Plugin Properties page
82+
Then Navigate to the properties page of plugin: "CloudSQL MySQL"
83+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "cloudsql-mysql"
84+
Then Select radio button plugin property: "instanceType" with value: "public"
85+
Then Enter input plugin property: "connectionName" with value: "ConnectionName"
86+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
87+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
88+
Then Enter input plugin property: "referenceName" with value: "RefName"
89+
Then Enter input plugin property: "database" with value: "DatabaseName"
90+
Then Click on the Macro button of Property: "tableName" and set the value to: "Tablename"
91+
Then Validate "CloudSQL MySQL2" plugin properties
92+
Then Close the Plugin Properties page
93+
Then Save the pipeline
94+
Then Preview and run the pipeline
95+
Then Enter runtime argument value "targetTable" for key "Tablename"
96+
Then Run the preview of pipeline with runtime arguments
97+
Then Wait till pipeline preview is in running state
98+
Then Open and capture pipeline preview logs
99+
Then Verify the preview run status of pipeline in the logs is "succeeded"
100+
Then Close the pipeline logs
101+
Then Close the preview
102+
Then Deploy the pipeline
103+
Then Run the Pipeline in Runtime
104+
Then Enter runtime argument value "targetTable" for key "Tablename"
105+
Then Run the Pipeline in Runtime with runtime arguments
106+
Then Wait till pipeline is in running state
107+
Then Open and capture logs
108+
Then Verify the pipeline status is "Succeeded"
109+
Then Close the pipeline logs
110+
111+
@BQ_SOURCE_TEST @CLOUDMYSQL_TEST_TABLE
112+
Scenario: Verify pipeline failure message in logs when user provides invalid Table Name of plugin with Macros
113+
Given Open Datafusion Project to configure pipeline
114+
When Expand Plugin group in the LHS plugins list: "Source"
115+
When Select plugin: "BigQuery" from the plugins list as: "Source"
116+
When Expand Plugin group in the LHS plugins list: "Sink"
117+
When Select plugin: "CloudSQL MySQL" from the plugins list as: "Sink"
118+
Then Connect plugins: "BigQuery" and "CloudSQL MySQL" to establish connection
119+
Then Navigate to the properties page of plugin: "BigQuery"
120+
And Replace input plugin property: "project" with value: "projectId"
121+
And Enter input plugin property: "datasetProject" with value: "datasetprojectId"
122+
And Enter input plugin property: "dataset" with value: "dataset"
123+
And Enter input plugin property: "table" with value: "bqSourceTable"
124+
Then Validate "BigQuery" plugin properties
125+
Then Close the Plugin Properties page
126+
Then Navigate to the properties page of plugin: "CloudSQL MySQL"
127+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "cloudsql-mysql"
128+
Then Select radio button plugin property: "instanceType" with value: "public"
129+
Then Enter input plugin property: "connectionName" with value: "ConnectionName"
130+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
131+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
132+
Then Enter input plugin property: "referenceName" with value: "RefName"
133+
Then Enter input plugin property: "database" with value: "DatabaseName"
134+
Then Click on the Macro button of Property: "tableName" and set the value to: "invalidTablename"
135+
Then Validate "CloudSQL MySQL2" plugin properties
136+
Then Close the Plugin Properties page
137+
Then Save the pipeline
138+
Then Deploy the pipeline
139+
Then Run the Pipeline in Runtime
140+
Then Enter runtime argument value "invalidTablename" for key "invalidTablename"
141+
Then Run the Pipeline in Runtime with runtime arguments
142+
Then Wait till pipeline is in running state
143+
Then Open and capture logs
144+
And Verify the pipeline status is "Failed"
145+
146+
@BQ_SOURCE_TEST @CLOUDMYSQL_TEST_TABLE
147+
Scenario: Verify pipeline failure message in logs when user provides invalid credentials of plugin with Macros
148+
Given Open Datafusion Project to configure pipeline
149+
When Expand Plugin group in the LHS plugins list: "Source"
150+
When Select plugin: "BigQuery" from the plugins list as: "Source"
151+
When Expand Plugin group in the LHS plugins list: "Sink"
152+
When Select plugin: "CloudSQL MySQL" from the plugins list as: "Sink"
153+
Then Connect plugins: "BigQuery" and "CloudSQL MySQL" to establish connection
154+
Then Navigate to the properties page of plugin: "BigQuery"
155+
Then Enter input plugin property: "referenceName" with value: "BQReferenceName"
156+
And Replace input plugin property: "project" with value: "projectId"
157+
And Enter input plugin property: "datasetProject" with value: "datasetprojectId"
158+
And Enter input plugin property: "dataset" with value: "dataset"
159+
And Enter input plugin property: "table" with value: "bqSourceTable"
160+
Then Validate "BigQuery" plugin properties
161+
Then Close the Plugin Properties page
162+
Then Navigate to the properties page of plugin: "CloudSQL MySQL"
163+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "cloudsql-mysql"
164+
Then Select radio button plugin property: "instanceType" with value: "public"
165+
Then Enter input plugin property: "connectionName" with value: "ConnectionName"
166+
Then Click on the Macro button of Property: "user" and set the value to: "Username"
167+
Then Click on the Macro button of Property: "password" and set the value to: "Password"
168+
Then Enter input plugin property: "referenceName" with value: "RefName"
169+
Then Enter input plugin property: "database" with value: "DatabaseName"
170+
Then Replace input plugin property: "tableName" with value: "targetTable"
171+
Then Validate "CloudSQL MySQL" plugin properties
172+
Then Close the Plugin Properties page
173+
Then Save the pipeline
174+
Then Deploy the pipeline
175+
Then Run the Pipeline in Runtime
176+
Then Enter runtime argument value "invalidUserName" for key "Username"
177+
Then Enter runtime argument value "invalidPassword" for key "Password"
178+
Then Run the Pipeline in Runtime with runtime arguments
179+
Then Wait till pipeline is in running state
180+
Then Open and capture logs
181+
And Verify the pipeline status is "Failed"
182+

‎cloudsql-mysql-plugin/src/e2e-test/features/source/CloudMySqlRunTime.feature‎

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -229,6 +229,3 @@ Feature: CloudMySql - Verify data transfer from CloudMySql source to BigQuery si
229229
Then Wait till pipeline is in running state
230230
Then Open and capture logs
231231
And Verify the pipeline status is "Failed"
232-
233-
234-

0 commit comments

Comments
 (0)