You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@texera.apache.org by "gracecluvohio (via GitHub)" <gi...@apache.org> on 2025/09/18 10:13:17 UTC
[PR] feat: add a new choropleth map operator [texera]
gracecluvohio opened a new pull request, #3744:
URL: https://github.com/apache/texera/pull/3744
- Purpose of the PR:
- Texera doesn't have any map operators, limiting its visualization of geographical data. Choropleth maps use differences in shading or color to dynamically show differences in properties or quantities between regions. For example, below is a choropleth map showing life expectancy in 2007 across different countries (generated by my new operator!)
- <img width="651" height="269" alt="Screenshot 2025-09-18 at 3 07 03 AM" src="https://github.com/user-attachments/assets/0ebd78aa-818d-4688-8cdd-609c67cebe76" />
- Choropleth maps are simple to understand yet powerful. Because of this they are an excellent first step for geographical visualizations in Texera.
- Summary of changes:
- Adding ChoroplethMapOpDesc.scala with the operator logic
- Adding an icon for the operator
- Registering the operator in the logical and physical layers in the LogicalOp.scala
- Future Work
- adding option to change region level (right now it is by country, but could add by state for US-specific)
- adding option to focus on specific region
- Screenshots
Choropleth map UI
<img width="1241" height="913" alt="Screenshot 2025-09-18 at 2 57 05 AM" src="https://github.com/user-attachments/assets/eda2df3e-75f3-42f5-a942-82d1bf60981a" />
Results
<img width="1238" height="914" alt="Screenshot 2025-09-18 at 2 56 10 AM" src="https://github.com/user-attachments/assets/0f1d8579-e12f-4a71-a249-e59f52552cc5" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: add a new choropleth map operator [texera]
Posted by "gracecluvohio (via GitHub)" <gi...@apache.org>.
gracecluvohio commented on code in PR #3744:
URL: https://github.com/apache/texera/pull/3744#discussion_r2364618331
##########
core/workflow-operator/src/main/scala/edu/uci/ics/amber/operator/visualization/choroplethMap/ChoroplethMapOpDesc.scala:
##########
@@ -0,0 +1,112 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package edu.uci.ics.amber.operator.visualization.choroplethMap
+
+import com.fasterxml.jackson.annotation.{JsonProperty, JsonPropertyDescription}
+import com.kjetland.jackson.jsonSchema.annotations.JsonSchemaTitle
+import edu.uci.ics.amber.core.tuple.{AttributeType, Schema}
+import edu.uci.ics.amber.operator.PythonOperatorDescriptor
+import edu.uci.ics.amber.core.workflow.OutputPort.OutputMode
+import edu.uci.ics.amber.core.workflow.{InputPort, OutputPort, PortIdentity}
+import edu.uci.ics.amber.operator.metadata.{OperatorGroupConstants, OperatorInfo}
+import edu.uci.ics.amber.operator.metadata.annotations.AutofillAttributeName
+
Review Comment:
Ok, I'll add for both color and locations fields.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: add a new choropleth map operator [texera]
Posted by "gracecluvohio (via GitHub)" <gi...@apache.org>.
gracecluvohio commented on code in PR #3744:
URL: https://github.com/apache/texera/pull/3744#discussion_r2361426924
##########
core/workflow-operator/src/main/scala/edu/uci/ics/amber/operator/visualization/choroplethMap/ChoroplethMapOpDesc.scala:
##########
@@ -0,0 +1,113 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package edu.uci.ics.amber.operator.visualization.choroplethMap
+
+import com.fasterxml.jackson.annotation.{JsonProperty, JsonPropertyDescription}
+import com.kjetland.jackson.jsonSchema.annotations.JsonSchemaTitle
+import edu.uci.ics.amber.core.tuple.{AttributeType, Schema}
+import edu.uci.ics.amber.operator.PythonOperatorDescriptor
+import edu.uci.ics.amber.core.workflow.OutputPort.OutputMode
+import edu.uci.ics.amber.core.workflow.{InputPort, OutputPort, PortIdentity}
+import edu.uci.ics.amber.operator.metadata.{OperatorGroupConstants, OperatorInfo}
+import edu.uci.ics.amber.operator.metadata.annotations.AutofillAttributeName
+
+class ChoroplethMapOpDesc extends PythonOperatorDescriptor {
+
+ @JsonProperty(value = "locations", required = true)
+ @JsonSchemaTitle("Locations Column")
+ @JsonPropertyDescription("what column is used to describe the locations")
+ @AutofillAttributeName
+ var locations: String = ""
+
+ @JsonProperty(value = "color", required = true)
+ @JsonSchemaTitle("Color Column")
+ @JsonPropertyDescription(
+ "what column is used to determine intensity of color of the state/country"
+ )
+ @AutofillAttributeName
+ var color: String = ""
+
+ override def getOutputSchemas(
+ inputSchemas: Map[PortIdentity, Schema]
+ ): Map[PortIdentity, Schema] = {
+ val outputSchema = Schema()
+ .add("html-content", AttributeType.STRING)
+ Map(operatorInfo.outputPorts.head.id -> outputSchema)
+ Map(operatorInfo.outputPorts.head.id -> outputSchema)
+ }
+
+ override def operatorInfo: OperatorInfo =
+ OperatorInfo(
+ "Choropleth Map",
+ "a Choropleth Map; higher color intensity indicates a higher statistic for that region",
+ OperatorGroupConstants.VISUALIZATION_ADVANCED_GROUP,
+ inputPorts = List(InputPort()),
+ outputPorts = List(OutputPort(mode = OutputMode.SINGLE_SNAPSHOT))
+ )
+
+ def manipulateTable(): String = {
+ assert(locations.nonEmpty)
+ assert(color.nonEmpty)
+ s"""
+ | table.dropna(subset=['$locations', '$color'], inplace = True)
+ |""".stripMargin
+ }
+
+ def createPlotlyFigure(): String = {
+ assert(locations.nonEmpty && color.nonEmpty)
+ s"""
+ | fig = px.choropleth(table, locations="country", color="2007", color_continuous_scale=px.colors.sequential.Plasma)
Review Comment:
Updated, thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: add a new choropleth map operator [texera]
Posted by "gracecluvohio (via GitHub)" <gi...@apache.org>.
gracecluvohio commented on code in PR #3744:
URL: https://github.com/apache/texera/pull/3744#discussion_r2364644236
##########
core/workflow-operator/src/main/scala/edu/uci/ics/amber/operator/visualization/choroplethMap/ChoroplethMapOpDesc.scala:
##########
@@ -0,0 +1,112 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package edu.uci.ics.amber.operator.visualization.choroplethMap
+
+import com.fasterxml.jackson.annotation.{JsonProperty, JsonPropertyDescription}
+import com.kjetland.jackson.jsonSchema.annotations.JsonSchemaTitle
+import edu.uci.ics.amber.core.tuple.{AttributeType, Schema}
+import edu.uci.ics.amber.operator.PythonOperatorDescriptor
+import edu.uci.ics.amber.core.workflow.OutputPort.OutputMode
+import edu.uci.ics.amber.core.workflow.{InputPort, OutputPort, PortIdentity}
+import edu.uci.ics.amber.operator.metadata.{OperatorGroupConstants, OperatorInfo}
+import edu.uci.ics.amber.operator.metadata.annotations.AutofillAttributeName
+
+class ChoroplethMapOpDesc extends PythonOperatorDescriptor {
+
+ @JsonProperty(value = "locations", required = true)
Review Comment:
Updated in PR and property description
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: add a new choropleth map operator [texera]
Posted by "xuang7 (via GitHub)" <gi...@apache.org>.
xuang7 commented on code in PR #3744:
URL: https://github.com/apache/texera/pull/3744#discussion_r2361324297
##########
core/workflow-operator/src/main/scala/edu/uci/ics/amber/operator/visualization/choroplethMap/ChoroplethMapOpDesc.scala:
##########
@@ -0,0 +1,113 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package edu.uci.ics.amber.operator.visualization.choroplethMap
+
+import com.fasterxml.jackson.annotation.{JsonProperty, JsonPropertyDescription}
+import com.kjetland.jackson.jsonSchema.annotations.JsonSchemaTitle
+import edu.uci.ics.amber.core.tuple.{AttributeType, Schema}
+import edu.uci.ics.amber.operator.PythonOperatorDescriptor
+import edu.uci.ics.amber.core.workflow.OutputPort.OutputMode
+import edu.uci.ics.amber.core.workflow.{InputPort, OutputPort, PortIdentity}
+import edu.uci.ics.amber.operator.metadata.{OperatorGroupConstants, OperatorInfo}
+import edu.uci.ics.amber.operator.metadata.annotations.AutofillAttributeName
+
+class ChoroplethMapOpDesc extends PythonOperatorDescriptor {
+
+ @JsonProperty(value = "locations", required = true)
+ @JsonSchemaTitle("Locations Column")
+ @JsonPropertyDescription("what column is used to describe the locations")
+ @AutofillAttributeName
+ var locations: String = ""
+
+ @JsonProperty(value = "color", required = true)
+ @JsonSchemaTitle("Color Column")
+ @JsonPropertyDescription(
+ "what column is used to determine intensity of color of the state/country"
+ )
+ @AutofillAttributeName
+ var color: String = ""
+
+ override def getOutputSchemas(
+ inputSchemas: Map[PortIdentity, Schema]
+ ): Map[PortIdentity, Schema] = {
+ val outputSchema = Schema()
+ .add("html-content", AttributeType.STRING)
+ Map(operatorInfo.outputPorts.head.id -> outputSchema)
+ Map(operatorInfo.outputPorts.head.id -> outputSchema)
+ }
+
+ override def operatorInfo: OperatorInfo =
+ OperatorInfo(
+ "Choropleth Map",
+ "a Choropleth Map; higher color intensity indicates a higher statistic for that region",
+ OperatorGroupConstants.VISUALIZATION_ADVANCED_GROUP,
+ inputPorts = List(InputPort()),
+ outputPorts = List(OutputPort(mode = OutputMode.SINGLE_SNAPSHOT))
+ )
+
+ def manipulateTable(): String = {
+ assert(locations.nonEmpty)
+ assert(color.nonEmpty)
+ s"""
+ | table.dropna(subset=['$locations', '$color'], inplace = True)
+ |""".stripMargin
+ }
+
+ def createPlotlyFigure(): String = {
+ assert(locations.nonEmpty && color.nonEmpty)
+ s"""
+ | fig = px.choropleth(table, locations="country", color="2007", color_continuous_scale=px.colors.sequential.Plasma)
Review Comment:
createPlotlyFigure() hard-codes country/2007 instead of using the configured $locations and $color.
##########
core/workflow-operator/src/main/scala/edu/uci/ics/amber/operator/visualization/choroplethMap/ChoroplethMapOpDesc.scala:
##########
@@ -0,0 +1,113 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package edu.uci.ics.amber.operator.visualization.choroplethMap
+
+import com.fasterxml.jackson.annotation.{JsonProperty, JsonPropertyDescription}
+import com.kjetland.jackson.jsonSchema.annotations.JsonSchemaTitle
+import edu.uci.ics.amber.core.tuple.{AttributeType, Schema}
+import edu.uci.ics.amber.operator.PythonOperatorDescriptor
+import edu.uci.ics.amber.core.workflow.OutputPort.OutputMode
+import edu.uci.ics.amber.core.workflow.{InputPort, OutputPort, PortIdentity}
+import edu.uci.ics.amber.operator.metadata.{OperatorGroupConstants, OperatorInfo}
+import edu.uci.ics.amber.operator.metadata.annotations.AutofillAttributeName
+
+class ChoroplethMapOpDesc extends PythonOperatorDescriptor {
+
+ @JsonProperty(value = "locations", required = true)
+ @JsonSchemaTitle("Locations Column")
+ @JsonPropertyDescription("what column is used to describe the locations")
Review Comment:
The first word should be capitalized here.
##########
core/workflow-operator/src/main/scala/edu/uci/ics/amber/operator/visualization/choroplethMap/ChoroplethMapOpDesc.scala:
##########
@@ -0,0 +1,113 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package edu.uci.ics.amber.operator.visualization.choroplethMap
+
+import com.fasterxml.jackson.annotation.{JsonProperty, JsonPropertyDescription}
+import com.kjetland.jackson.jsonSchema.annotations.JsonSchemaTitle
+import edu.uci.ics.amber.core.tuple.{AttributeType, Schema}
+import edu.uci.ics.amber.operator.PythonOperatorDescriptor
+import edu.uci.ics.amber.core.workflow.OutputPort.OutputMode
+import edu.uci.ics.amber.core.workflow.{InputPort, OutputPort, PortIdentity}
+import edu.uci.ics.amber.operator.metadata.{OperatorGroupConstants, OperatorInfo}
+import edu.uci.ics.amber.operator.metadata.annotations.AutofillAttributeName
+
+class ChoroplethMapOpDesc extends PythonOperatorDescriptor {
+
+ @JsonProperty(value = "locations", required = true)
+ @JsonSchemaTitle("Locations Column")
+ @JsonPropertyDescription("what column is used to describe the locations")
+ @AutofillAttributeName
+ var locations: String = ""
+
+ @JsonProperty(value = "color", required = true)
+ @JsonSchemaTitle("Color Column")
+ @JsonPropertyDescription(
+ "what column is used to determine intensity of color of the state/country"
Review Comment:
same.
##########
core/workflow-operator/src/main/scala/edu/uci/ics/amber/operator/visualization/choroplethMap/ChoroplethMapOpDesc.scala:
##########
@@ -0,0 +1,113 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package edu.uci.ics.amber.operator.visualization.choroplethMap
+
+import com.fasterxml.jackson.annotation.{JsonProperty, JsonPropertyDescription}
+import com.kjetland.jackson.jsonSchema.annotations.JsonSchemaTitle
+import edu.uci.ics.amber.core.tuple.{AttributeType, Schema}
+import edu.uci.ics.amber.operator.PythonOperatorDescriptor
+import edu.uci.ics.amber.core.workflow.OutputPort.OutputMode
+import edu.uci.ics.amber.core.workflow.{InputPort, OutputPort, PortIdentity}
+import edu.uci.ics.amber.operator.metadata.{OperatorGroupConstants, OperatorInfo}
+import edu.uci.ics.amber.operator.metadata.annotations.AutofillAttributeName
+
+class ChoroplethMapOpDesc extends PythonOperatorDescriptor {
+
+ @JsonProperty(value = "locations", required = true)
+ @JsonSchemaTitle("Locations Column")
+ @JsonPropertyDescription("what column is used to describe the locations")
+ @AutofillAttributeName
+ var locations: String = ""
+
+ @JsonProperty(value = "color", required = true)
+ @JsonSchemaTitle("Color Column")
+ @JsonPropertyDescription(
+ "what column is used to determine intensity of color of the state/country"
+ )
+ @AutofillAttributeName
+ var color: String = ""
+
+ override def getOutputSchemas(
+ inputSchemas: Map[PortIdentity, Schema]
+ ): Map[PortIdentity, Schema] = {
+ val outputSchema = Schema()
+ .add("html-content", AttributeType.STRING)
+ Map(operatorInfo.outputPorts.head.id -> outputSchema)
+ Map(operatorInfo.outputPorts.head.id -> outputSchema)
Review Comment:
Looks like this line is duplicated.
##########
core/workflow-operator/src/main/scala/edu/uci/ics/amber/operator/visualization/choroplethMap/ChoroplethMapOpDesc.scala:
##########
@@ -0,0 +1,113 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package edu.uci.ics.amber.operator.visualization.choroplethMap
+
+import com.fasterxml.jackson.annotation.{JsonProperty, JsonPropertyDescription}
+import com.kjetland.jackson.jsonSchema.annotations.JsonSchemaTitle
+import edu.uci.ics.amber.core.tuple.{AttributeType, Schema}
+import edu.uci.ics.amber.operator.PythonOperatorDescriptor
+import edu.uci.ics.amber.core.workflow.OutputPort.OutputMode
+import edu.uci.ics.amber.core.workflow.{InputPort, OutputPort, PortIdentity}
+import edu.uci.ics.amber.operator.metadata.{OperatorGroupConstants, OperatorInfo}
+import edu.uci.ics.amber.operator.metadata.annotations.AutofillAttributeName
+
+class ChoroplethMapOpDesc extends PythonOperatorDescriptor {
+
+ @JsonProperty(value = "locations", required = true)
+ @JsonSchemaTitle("Locations Column")
+ @JsonPropertyDescription("what column is used to describe the locations")
+ @AutofillAttributeName
+ var locations: String = ""
+
+ @JsonProperty(value = "color", required = true)
+ @JsonSchemaTitle("Color Column")
+ @JsonPropertyDescription(
+ "what column is used to determine intensity of color of the state/country"
+ )
+ @AutofillAttributeName
+ var color: String = ""
+
+ override def getOutputSchemas(
+ inputSchemas: Map[PortIdentity, Schema]
+ ): Map[PortIdentity, Schema] = {
+ val outputSchema = Schema()
+ .add("html-content", AttributeType.STRING)
+ Map(operatorInfo.outputPorts.head.id -> outputSchema)
+ Map(operatorInfo.outputPorts.head.id -> outputSchema)
+ }
+
+ override def operatorInfo: OperatorInfo =
+ OperatorInfo(
+ "Choropleth Map",
+ "a Choropleth Map; higher color intensity indicates a higher statistic for that region",
Review Comment:
Please match the description style used by the other operators.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: add a new choropleth map operator [texera]
Posted by "aglinxinyuan (via GitHub)" <gi...@apache.org>.
aglinxinyuan merged PR #3744:
URL: https://github.com/apache/texera/pull/3744
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: add a new choropleth map operator [texera]
Posted by "mengw15 (via GitHub)" <gi...@apache.org>.
mengw15 commented on code in PR #3744:
URL: https://github.com/apache/texera/pull/3744#discussion_r2364267811
##########
core/gui/src/assets/operator_images/ChoroplethMap.png:
##########
Review Comment:
Could you change it to a more representative icon? It's hard for me to tell that this icon is an operator about map.
##########
core/workflow-operator/src/main/scala/edu/uci/ics/amber/operator/visualization/choroplethMap/ChoroplethMapOpDesc.scala:
##########
@@ -0,0 +1,112 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package edu.uci.ics.amber.operator.visualization.choroplethMap
+
+import com.fasterxml.jackson.annotation.{JsonProperty, JsonPropertyDescription}
+import com.kjetland.jackson.jsonSchema.annotations.JsonSchemaTitle
+import edu.uci.ics.amber.core.tuple.{AttributeType, Schema}
+import edu.uci.ics.amber.operator.PythonOperatorDescriptor
+import edu.uci.ics.amber.core.workflow.OutputPort.OutputMode
+import edu.uci.ics.amber.core.workflow.{InputPort, OutputPort, PortIdentity}
+import edu.uci.ics.amber.operator.metadata.{OperatorGroupConstants, OperatorInfo}
+import edu.uci.ics.amber.operator.metadata.annotations.AutofillAttributeName
+
Review Comment:
consider adding something to constraint column:
example:
@JsonSchemaInject(json = """
{
"attributeTypeRules": {
"color": {
"enum": ["integer", "long", "double"]
}
}
}
""")
##########
core/workflow-operator/src/main/scala/edu/uci/ics/amber/operator/visualization/choroplethMap/ChoroplethMapOpDesc.scala:
##########
@@ -0,0 +1,112 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package edu.uci.ics.amber.operator.visualization.choroplethMap
+
+import com.fasterxml.jackson.annotation.{JsonProperty, JsonPropertyDescription}
+import com.kjetland.jackson.jsonSchema.annotations.JsonSchemaTitle
+import edu.uci.ics.amber.core.tuple.{AttributeType, Schema}
+import edu.uci.ics.amber.operator.PythonOperatorDescriptor
+import edu.uci.ics.amber.core.workflow.OutputPort.OutputMode
+import edu.uci.ics.amber.core.workflow.{InputPort, OutputPort, PortIdentity}
+import edu.uci.ics.amber.operator.metadata.{OperatorGroupConstants, OperatorInfo}
+import edu.uci.ics.amber.operator.metadata.annotations.AutofillAttributeName
+
+class ChoroplethMapOpDesc extends PythonOperatorDescriptor {
+
+ @JsonProperty(value = "locations", required = true)
+ @JsonSchemaTitle("Locations Column")
+ @JsonPropertyDescription("Column used to describe location")
+ @AutofillAttributeName
+ var locations: String = ""
+
+ @JsonProperty(value = "color", required = true)
+ @JsonSchemaTitle("Color Column")
+ @JsonPropertyDescription(
+ "Column used to determine intensity of color of the region"
+ )
+ @AutofillAttributeName
+ var color: String = ""
+
+ override def getOutputSchemas(
+ inputSchemas: Map[PortIdentity, Schema]
+ ): Map[PortIdentity, Schema] = {
+ val outputSchema = Schema()
+ .add("html-content", AttributeType.STRING)
+ Map(operatorInfo.outputPorts.head.id -> outputSchema)
+ }
+
+ override def operatorInfo: OperatorInfo =
+ OperatorInfo(
+ "Choropleth Map",
+ "Visualize data using a Choropleth Map that uses shades of colors to show differences in properties or quantities between regions",
+ OperatorGroupConstants.VISUALIZATION_ADVANCED_GROUP,
+ inputPorts = List(InputPort()),
+ outputPorts = List(OutputPort(mode = OutputMode.SINGLE_SNAPSHOT))
+ )
+
+ def manipulateTable(): String = {
+ assert(locations.nonEmpty)
+ assert(color.nonEmpty)
+ s"""
+ | table.dropna(subset=['$locations', '$color'], inplace = True)
+ |""".stripMargin
+ }
+
+ def createPlotlyFigure(): String = {
+ assert(locations.nonEmpty && color.nonEmpty)
+ s"""
+ | fig = px.choropleth(table, locations="$locations", color="$color", color_continuous_scale=px.colors.sequential.Plasma)
+ |""".stripMargin
+ }
+
+ override def generatePythonCode(): String = {
+ val finalCode =
+ s"""
+ |from pytexera import *
+ |
+ |import plotly.express as px
+ |import plotly.io
+ |import plotly
+ |
+ |class ProcessTableOperator(UDFTableOperator):
+ |
+ | # Generate custom error message as html string
+ | def render_error(self, error_msg) -> str:
+ | return '''<h1>Choropleth map is not available.</h1>
+ | <p>Reason is: {} </p>
+ | '''.format(error_msg)
+ |
+ | @overrides
+ | def process_table(self, table: Table, port: int) -> Iterator[Optional[TableLike]]:
+ | if table.empty:
+ | yield {'html-content': self.render_error("Input table is empty.")}
+ | return
+ | ${manipulateTable()}
+ | if table.empty:
+ | yield {'html-content': self.render_error("No valid rows left (every row has at least 1 missing value).")}
+ | return
+ | ${createPlotlyFigure()}
+ | fig.update_layout(margin={"r":0,"t":0,"l":0,"b":0})
Review Comment:
consider move this line in to the createPlotlyFigure() function
##########
core/workflow-operator/src/main/scala/edu/uci/ics/amber/operator/visualization/choroplethMap/ChoroplethMapOpDesc.scala:
##########
@@ -0,0 +1,112 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package edu.uci.ics.amber.operator.visualization.choroplethMap
+
+import com.fasterxml.jackson.annotation.{JsonProperty, JsonPropertyDescription}
+import com.kjetland.jackson.jsonSchema.annotations.JsonSchemaTitle
+import edu.uci.ics.amber.core.tuple.{AttributeType, Schema}
+import edu.uci.ics.amber.operator.PythonOperatorDescriptor
+import edu.uci.ics.amber.core.workflow.OutputPort.OutputMode
+import edu.uci.ics.amber.core.workflow.{InputPort, OutputPort, PortIdentity}
+import edu.uci.ics.amber.operator.metadata.{OperatorGroupConstants, OperatorInfo}
+import edu.uci.ics.amber.operator.metadata.annotations.AutofillAttributeName
+
+class ChoroplethMapOpDesc extends PythonOperatorDescriptor {
+
+ @JsonProperty(value = "locations", required = true)
Review Comment:
it seems to me that there are constraints about the location column format? Could you elaborate the constraint in pr description and give some example(like format need to be "USA CHN JPN"). and somehow to notify user in the property description?
##########
core/workflow-operator/src/main/scala/edu/uci/ics/amber/operator/visualization/choroplethMap/ChoroplethMapOpDesc.scala:
##########
@@ -0,0 +1,112 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package edu.uci.ics.amber.operator.visualization.choroplethMap
+
+import com.fasterxml.jackson.annotation.{JsonProperty, JsonPropertyDescription}
+import com.kjetland.jackson.jsonSchema.annotations.JsonSchemaTitle
+import edu.uci.ics.amber.core.tuple.{AttributeType, Schema}
+import edu.uci.ics.amber.operator.PythonOperatorDescriptor
+import edu.uci.ics.amber.core.workflow.OutputPort.OutputMode
+import edu.uci.ics.amber.core.workflow.{InputPort, OutputPort, PortIdentity}
+import edu.uci.ics.amber.operator.metadata.{OperatorGroupConstants, OperatorInfo}
+import edu.uci.ics.amber.operator.metadata.annotations.AutofillAttributeName
+
+class ChoroplethMapOpDesc extends PythonOperatorDescriptor {
+
+ @JsonProperty(value = "locations", required = true)
+ @JsonSchemaTitle("Locations Column")
+ @JsonPropertyDescription("Column used to describe location")
Review Comment:
in your pr description, it says "right now it is by country", could you also add the information for users? and also mention, if the country value need to be in certain format.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: add a new choropleth map operator [texera]
Posted by "gracecluvohio (via GitHub)" <gi...@apache.org>.
gracecluvohio commented on code in PR #3744:
URL: https://github.com/apache/texera/pull/3744#discussion_r2361417895
##########
core/workflow-operator/src/main/scala/edu/uci/ics/amber/operator/visualization/choroplethMap/ChoroplethMapOpDesc.scala:
##########
@@ -0,0 +1,113 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package edu.uci.ics.amber.operator.visualization.choroplethMap
+
+import com.fasterxml.jackson.annotation.{JsonProperty, JsonPropertyDescription}
+import com.kjetland.jackson.jsonSchema.annotations.JsonSchemaTitle
+import edu.uci.ics.amber.core.tuple.{AttributeType, Schema}
+import edu.uci.ics.amber.operator.PythonOperatorDescriptor
+import edu.uci.ics.amber.core.workflow.OutputPort.OutputMode
+import edu.uci.ics.amber.core.workflow.{InputPort, OutputPort, PortIdentity}
+import edu.uci.ics.amber.operator.metadata.{OperatorGroupConstants, OperatorInfo}
+import edu.uci.ics.amber.operator.metadata.annotations.AutofillAttributeName
+
+class ChoroplethMapOpDesc extends PythonOperatorDescriptor {
+
+ @JsonProperty(value = "locations", required = true)
+ @JsonSchemaTitle("Locations Column")
+ @JsonPropertyDescription("what column is used to describe the locations")
+ @AutofillAttributeName
+ var locations: String = ""
+
+ @JsonProperty(value = "color", required = true)
+ @JsonSchemaTitle("Color Column")
+ @JsonPropertyDescription(
+ "what column is used to determine intensity of color of the state/country"
+ )
+ @AutofillAttributeName
+ var color: String = ""
+
+ override def getOutputSchemas(
+ inputSchemas: Map[PortIdentity, Schema]
+ ): Map[PortIdentity, Schema] = {
+ val outputSchema = Schema()
+ .add("html-content", AttributeType.STRING)
+ Map(operatorInfo.outputPorts.head.id -> outputSchema)
+ Map(operatorInfo.outputPorts.head.id -> outputSchema)
Review Comment:
Good catch!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: add a new choropleth map operator [texera]
Posted by "gracecluvohio (via GitHub)" <gi...@apache.org>.
gracecluvohio commented on code in PR #3744:
URL: https://github.com/apache/texera/pull/3744#discussion_r2364649732
##########
core/workflow-operator/src/main/scala/edu/uci/ics/amber/operator/visualization/choroplethMap/ChoroplethMapOpDesc.scala:
##########
@@ -0,0 +1,112 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package edu.uci.ics.amber.operator.visualization.choroplethMap
+
+import com.fasterxml.jackson.annotation.{JsonProperty, JsonPropertyDescription}
+import com.kjetland.jackson.jsonSchema.annotations.JsonSchemaTitle
+import edu.uci.ics.amber.core.tuple.{AttributeType, Schema}
+import edu.uci.ics.amber.operator.PythonOperatorDescriptor
+import edu.uci.ics.amber.core.workflow.OutputPort.OutputMode
+import edu.uci.ics.amber.core.workflow.{InputPort, OutputPort, PortIdentity}
+import edu.uci.ics.amber.operator.metadata.{OperatorGroupConstants, OperatorInfo}
+import edu.uci.ics.amber.operator.metadata.annotations.AutofillAttributeName
+
+class ChoroplethMapOpDesc extends PythonOperatorDescriptor {
+
+ @JsonProperty(value = "locations", required = true)
+ @JsonSchemaTitle("Locations Column")
+ @JsonPropertyDescription("Column used to describe location")
+ @AutofillAttributeName
+ var locations: String = ""
+
+ @JsonProperty(value = "color", required = true)
+ @JsonSchemaTitle("Color Column")
+ @JsonPropertyDescription(
+ "Column used to determine intensity of color of the region"
+ )
+ @AutofillAttributeName
+ var color: String = ""
+
+ override def getOutputSchemas(
+ inputSchemas: Map[PortIdentity, Schema]
+ ): Map[PortIdentity, Schema] = {
+ val outputSchema = Schema()
+ .add("html-content", AttributeType.STRING)
+ Map(operatorInfo.outputPorts.head.id -> outputSchema)
+ }
+
+ override def operatorInfo: OperatorInfo =
+ OperatorInfo(
+ "Choropleth Map",
+ "Visualize data using a Choropleth Map that uses shades of colors to show differences in properties or quantities between regions",
+ OperatorGroupConstants.VISUALIZATION_ADVANCED_GROUP,
+ inputPorts = List(InputPort()),
+ outputPorts = List(OutputPort(mode = OutputMode.SINGLE_SNAPSHOT))
+ )
+
+ def manipulateTable(): String = {
+ assert(locations.nonEmpty)
+ assert(color.nonEmpty)
+ s"""
+ | table.dropna(subset=['$locations', '$color'], inplace = True)
+ |""".stripMargin
+ }
+
+ def createPlotlyFigure(): String = {
+ assert(locations.nonEmpty && color.nonEmpty)
+ s"""
+ | fig = px.choropleth(table, locations="$locations", color="$color", color_continuous_scale=px.colors.sequential.Plasma)
+ |""".stripMargin
+ }
+
+ override def generatePythonCode(): String = {
+ val finalCode =
+ s"""
+ |from pytexera import *
+ |
+ |import plotly.express as px
+ |import plotly.io
+ |import plotly
+ |
+ |class ProcessTableOperator(UDFTableOperator):
+ |
+ | # Generate custom error message as html string
+ | def render_error(self, error_msg) -> str:
+ | return '''<h1>Choropleth map is not available.</h1>
+ | <p>Reason is: {} </p>
+ | '''.format(error_msg)
+ |
+ | @overrides
+ | def process_table(self, table: Table, port: int) -> Iterator[Optional[TableLike]]:
+ | if table.empty:
+ | yield {'html-content': self.render_error("Input table is empty.")}
+ | return
+ | ${manipulateTable()}
+ | if table.empty:
+ | yield {'html-content': self.render_error("No valid rows left (every row has at least 1 missing value).")}
+ | return
+ | ${createPlotlyFigure()}
+ | fig.update_layout(margin={"r":0,"t":0,"l":0,"b":0})
Review Comment:
Sounds good
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: add a new choropleth map operator [texera]
Posted by "chenlica (via GitHub)" <gi...@apache.org>.
chenlica commented on PR #3744:
URL: https://github.com/apache/texera/pull/3744#issuecomment-3310772821
@xuang7 after you review and approve it, notify @mengw15 and @aglinxinyuan to review it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org