You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@heron.apache.org by sa...@apache.org on 2022/05/15 15:48:56 UTC
[incubator-heron] branch saadurrahman/native-python-rules-phase1 updated (f9a5f1877a1 -> 3ed8de94369)
This is an automated email from the ASF dual-hosted git repository.
saadurrahman pushed a change to branch saadurrahman/native-python-rules-phase1
in repository https://gitbox.apache.org/repos/asf/incubator-heron.git
from f9a5f1877a1 [Build] Workspace rules for protobuf
add 485ac317330 Fixing a type which broke the helm chart (#3775)
add ebd7ceaeb7c Update to fix Heron UI Jinja2 issue (#3777)
add 1a544770a66 Added missing packing plan handler in heron-tracker (#3783)
add 2baa27cdf68 Removing old Tornado asynchronous annotations from Heron shell (#3781)
add b7f6f3c51b4 Add back legacy CentOS 7 with newer gcc and Python (#3772)
add eacedb55ad2 Add Rocky 8 container option (#3773)
add 4f27c18e4d8 Fix conflict in Rocky 8 Dockerfile (#3787)
add a06dcd495ea Remove use of unsecure git protocol in build dependency (#3794)
add ef20cf51eda Remove unnecessary Bookkeeper format init container from Helm chart (#3795)
add da52c12d0da Change Log4J to SLF4J for Java Logging (#3791)
add b12a3965f4a Remove fixed young generation heap space of instance (#3789)
add 9cfc5c4785b Update to Guice 5.1.0 to support JDK17 (#3792)
add 3bca1c043bf Update build-docker.sh (#3799)
add bcaa38d2f42 Upgrade protobuf to 3.16.1 CVE-2021-22569 (#3797)
add 8841d1c4d6b Added Bazel Platform auto discovery (and lots of other related cleanup) (#3779)
add 7ee6160a260 Removing prevously missed refs to removed files (#3805)
add a68931e82a5 Updated to fix MacOS cppcheck RuntimeError (#3803)
add af7053ec816 Update cppcheck to 2.7 (#3804)
add bfdccb30d47 Updated Bazel rules_pkg build dependency to 0.6.0 (#3806)
add b7d92a99a08 Add logback.xml (#3807)
add 06e9f75988f Add SLF4JBridgeHandler (#3809)
add f353404d2ff Bump minimist from 1.2.5 to 1.2.6 in /website2/website (#3811)
add 547b7d9cbd6 Replace Helm chart's generic Bookkeeper platform property with specific properties (#3798)
add ae4099f15d8 Bump lodash from 4.17.15 to 4.17.21 in /website2/website (#3812)
add 700125f271b Updates to Heron UI and Tracker code to work with Python 3 updates (#3786)
add 909f60a4cf0 Bump url-parse from 1.4.7 to 1.5.10 in /website2/website (#3813)
add f07203a33b1 Remove ByteSize for protobuf (#3816)
add fd6a5fa0525 Bump prismjs from 1.19.0 to 1.27.0 in /website2/website (#3815)
add 7db7c24733b Support legacy API of HealthManger (#3818)
add 3fdf1f86071 Fix HealthManager (#3819)
add 6de42023e08 Bump async from 2.6.3 to 2.6.4 in /website2/website (#3822)
add b53b7fa97e7 Updated Netty to 4.1.76.Final (#3823)
add 5fac487bc11 Upgrade Kryo to 5.3.0 (#3824)
add c12b3c675a5 Publish unshaded Maven artifacts with dependencies (#3778)
add 820679cfa01 Added License file to published Jars (#3827)
add c18ff1b108f Update Debian Docker images from 10 to 11 (#3828)
add dc2d9f675a8 [3821] Remove deprecated Host Path, NFS, and EBS support for Kubernetes (#3825)
add 8d38e0e501f Update rules-scala of Scala (#3832)
add 3ed8de94369 Merge branch 'master' into saadurrahman/native-python-rules-phase1
No new revisions were added by this update.
Summary of changes:
LICENSE | 1 -
WORKSPACE | 120 +-
bazel_configure.py | 44 +-
contrib/bolts/kafka/src/java/BUILD | 1 +
contrib/spouts/kafka/src/java/BUILD | 1 +
deploy/kubernetes/helm/Chart.yaml.template | 2 +-
deploy/kubernetes/helm/templates/bookie.yaml | 31 +-
deploy/kubernetes/helm/values.yaml.template | 16 +-
...file.base.debian10 => Dockerfile.base.debian11} | 2 +-
.../{Dockerfile.centos8 => Dockerfile.centos7} | 28 +-
.../{Dockerfile.debian10 => Dockerfile.debian11} | 7 +-
.../{Dockerfile.centos8 => Dockerfile.rocky8} | 14 +-
docker/compile/Dockerfile.ubuntu18.04 | 7 +-
docker/compile/Dockerfile.ubuntu20.04 | 7 +-
docker/compile/Dockerfile.ubuntu22.04 | 8 +-
...erfile.dist.centos8 => Dockerfile.dist.centos7} | 12 +-
...file.dist.debian10 => Dockerfile.dist.debian11} | 2 +-
...kerfile.dist.centos8 => Dockerfile.dist.rocky8} | 14 +-
docker/dist/Dockerfile.dist.ubuntu20.04 | 1 +
...ist.ubuntu20.04 => Dockerfile.dist.ubuntu22.04} | 3 +-
docker/scripts/build-artifacts.sh | 2 +-
docker/scripts/build-base.sh | 4 +-
docker/scripts/build-docker.sh | 3 +-
docker/scripts/build-exec-docker.sh | 2 +-
docker/scripts/ci-docker.sh | 2 +-
docker/scripts/compile-docker.sh | 5 -
docker/scripts/compile-platform.sh | 19 +-
docker/scripts/dev-env-create.sh | 7 +-
docker/scripts/test-docker.sh | 5 -
docker/scripts/test-platform.sh | 16 +-
docker/scripts/test-unittest.sh | 4 +-
.../{Dockerfile.centos8 => Dockerfile.centos7} | 24 +-
.../test/{Dockerfile.centos8 => Dockerfile.rocky8} | 11 +-
docker/test/Dockerfile.ubuntu18.04 | 5 +-
eco/src/java/org/apache/heron/eco/Eco.java | 2 +
.../apache/heron/examples/api/AckingTopology.java | 12 +-
examples/src/python/bolt/consume_bolt.py | 2 +-
examples/src/python/bolt/count_bolt.py | 8 +-
examples/src/python/bolt/half_ack_bolt.py | 6 +-
examples/src/python/bolt/stateful_count_bolt.py | 8 +-
examples/src/python/bolt/stream_aggregate_bolt.py | 2 +-
examples/src/python/bolt/window_size_bolt.py | 4 +-
examples/src/python/spout/multi_stream_spout.py | 6 +-
examples/src/python/spout/stateful_word_spout.py | 6 +-
examples/src/python/spout/word_spout.py | 8 +-
examples/src/scala/compile.sh | 2 +-
heron/api/src/java/BUILD | 2 +
.../org/apache/heron/api/utils/Slf4jUtils.java | 17 +-
heron/api/src/java/shade.conf | 3 +
.../apache/heron/ckptmgr/CheckpointManager.java | 2 +
heron/ckptmgr/src/java/shade.conf | 8 +-
heron/common/src/cpp/basics/BUILD | 2 +-
heron/common/src/cpp/metrics/BUILD | 2 +-
heron/common/src/cpp/network/client.cpp | 12 +-
heron/common/src/cpp/network/event_loop_impl.cpp | 1 +
heron/common/src/cpp/network/server.cpp | 8 +-
heron/common/src/python/pex_loader.py | 22 +-
heron/common/tests/cpp/network/packet_unittest.cpp | 4 +-
heron/downloaders/src/java/BUILD | 1 +
.../apache/heron/downloader/DownloadRunner.java | 2 +
heron/executor/src/python/heron_executor.py | 294 +-
.../tests/python/heron_executor_unittest.py | 48 +-
heron/healthmgr/src/java/BUILD | 1 +
.../sensors/MetricsCacheMetricsProvider.java | 5 +-
.../healthmgr/sensors/TrackerMetricsProvider.java | 7 +-
heron/healthmgr/tests/java/BUILD | 1 +
.../cpp/boltimpl/bolt-output-collector-impl.cpp | 4 +-
heron/instance/src/cpp/gateway/stmgr-client.cpp | 6 +-
heron/instance/src/java/shade.conf | 2 +
heron/instance/src/python/basics/base_instance.py | 14 +-
heron/instance/src/python/basics/bolt_instance.py | 22 +-
heron/instance/src/python/basics/spout_instance.py | 26 +-
heron/instance/src/python/instance.py | 27 +-
.../instance/src/python/network/gateway_looper.py | 5 +-
heron/instance/src/python/network/heron_client.py | 48 +-
.../src/python/network/metricsmgr_client.py | 14 +-
heron/instance/src/python/network/protocol.py | 9 +-
.../instance/src/python/network/socket_options.py | 4 +-
.../src/python/utils/metrics/metrics_helper.py | 18 +-
.../src/python/utils/metrics/py_metrics.py | 2 +-
.../instance/src/python/utils/misc/communicator.py | 12 +-
.../python/utils/misc/custom_grouping_helper.py | 12 +-
.../instance/src/python/utils/misc/pplan_helper.py | 19 +-
.../src/python/utils/misc/serializer_helper.py | 6 +-
.../python/utils/topology/topology_context_impl.py | 20 +-
.../instance/tests/python/utils/mock_generator.py | 2 +-
heron/io/dlog/src/java/BUILD | 1 +
heron/io/dlog/tests/java/BUILD | 1 +
.../heron/metricscachemgr/MetricsCacheManager.java | 2 +
.../MetricsCacheManagerHttpServer.java | 2 +
.../apache/heron/metricsmgr/MetricsManager.java | 2 +
heron/proto/BUILD | 2 +-
.../apache/heron/scheduler/RuntimeManagerMain.java | 3 +-
.../org/apache/heron/scheduler/SchedulerMain.java | 3 +-
.../org/apache/heron/scheduler/SubmitterMain.java | 2 +
heron/schedulers/src/java/BUILD | 2 +-
.../scheduler/kubernetes/KubernetesContext.java | 51 -
.../heron/scheduler/kubernetes/V1Controller.java | 184 +-
.../apache/heron/scheduler/kubernetes/Volumes.java | 255 +-
heron/schedulers/tests/java/BUILD | 2 +-
.../scheduler/kubernetes/V1ControllerTest.java | 163 +-
.../heron/scheduler/kubernetes/VolumesTests.java | 350 +-
heron/shell/src/python/handlers/browsehandler.py | 9 +-
heron/shell/src/python/handlers/downloadhandler.py | 7 +-
heron/shell/src/python/handlers/filedatahandler.py | 8 +-
heron/shell/src/python/handlers/filehandler.py | 12 +-
.../shell/src/python/handlers/filestatshandler.py | 9 +-
heron/shell/src/python/handlers/healthhandler.py | 4 +-
heron/shell/src/python/handlers/jmaphandler.py | 6 +-
heron/shell/src/python/handlers/jstackhandler.py | 6 +-
.../src/python/handlers/killexecutorhandler.py | 7 +-
.../src/python/handlers/memoryhistogramhandler.py | 6 +-
heron/shell/src/python/handlers/pidhandler.py | 17 +-
heron/shell/src/python/handlers/pmaphandler.py | 6 +-
heron/shell/src/python/main.py | 2 +-
heron/shell/src/python/utils.py | 30 +-
heron/simulator/src/java/BUILD | 1 +
heron/simulator/src/java/shade.conf | 3 +
heron/spi/src/java/BUILD | 1 +
heron/statefulstorages/tests/java/BUILD | 2 +-
.../localfs/LocalFileSystemStateManager.java | 2 +
.../zookeeper/curator/CuratorStateManager.java | 3 +-
heron/statemgrs/src/python/configloader.py | 9 +-
heron/statemgrs/src/python/filestatemanager.py | 1 +
heron/statemgrs/src/python/statemanager.py | 7 +-
heron/statemgrs/src/python/statemanagerfactory.py | 2 +-
heron/statemgrs/src/python/zkstatemanager.py | 32 +-
heron/stmgr/src/cpp/manager/stmgr-client.cpp | 4 +-
heron/stmgr/src/cpp/manager/stmgr-server.cpp | 2 +-
heron/stmgr/src/cpp/manager/stmgr.cpp | 2 +-
heron/stmgr/src/cpp/util/tuple-cache.cpp | 8 +-
.../cpp/server/checkpoint-gateway_unittest.cpp | 6 +-
heron/stmgr/tests/cpp/server/stmgr_unittest.cpp | 2 +-
.../java/org/apache/heron/apiserver/Runtime.java | 2 +
heron/tools/cli/src/python/activate.py | 2 +-
heron/tools/cli/src/python/args.py | 11 +-
heron/tools/cli/src/python/cdefs.py | 6 +-
heron/tools/cli/src/python/cli_helper.py | 24 +-
heron/tools/cli/src/python/cliconfig.py | 8 +-
heron/tools/cli/src/python/config.py | 14 +-
heron/tools/cli/src/python/deactivate.py | 2 +-
heron/tools/cli/src/python/execute.py | 18 +-
heron/tools/cli/src/python/help.py | 2 +-
heron/tools/cli/src/python/jars.py | 2 +-
heron/tools/cli/src/python/kill.py | 2 +-
heron/tools/cli/src/python/main.py | 60 +-
heron/tools/cli/src/python/opts.py | 6 +-
heron/tools/cli/src/python/restart.py | 6 +-
heron/tools/cli/src/python/result.py | 19 +-
heron/tools/cli/src/python/submit.py | 56 +-
heron/tools/cli/src/python/update.py | 21 +-
heron/tools/cli/src/python/version.py | 12 +-
heron/tools/cli/tests/python/opts_unittest.py | 8 +-
heron/tools/common/src/python/clients/tracker.py | 76 +-
heron/tools/common/src/python/utils/config.py | 57 +-
heron/tools/explorer/src/python/logicalplan.py | 8 +-
heron/tools/explorer/src/python/main.py | 6 +-
heron/tools/explorer/src/python/physicalplan.py | 6 +-
heron/tools/explorer/src/python/topologies.py | 6 +-
heron/tools/tracker/src/python/BUILD | 6 +-
heron/tools/tracker/src/python/app.py | 52 +-
heron/tools/tracker/src/python/config.py | 1 +
heron/tools/tracker/src/python/main.py | 8 +-
heron/tools/tracker/src/python/metricstimeline.py | 32 +-
heron/tools/tracker/src/python/query_operators.py | 30 +-
.../tools/tracker/src/python/routers/container.py | 134 +-
heron/tools/tracker/src/python/routers/metrics.py | 76 +-
.../tools/tracker/src/python/routers/topologies.py | 23 +-
heron/tools/tracker/src/python/topology.py | 30 +-
heron/tools/tracker/src/python/tracker.py | 6 +-
heron/tools/tracker/src/python/utils.py | 82 +-
heron/tools/tracker/tests/python/app_unittest.py | 15 +-
.../tests/python/query_operator_unittest.py | 28 +-
heron/tools/ui/resources/static/js/exceptions.js | 14 +-
.../tools/ui/resources/static/js/physical-plan.js | 2 +-
.../ui/resources/static/js/plan-controller.js | 2 +-
heron/tools/ui/resources/static/js/topologies.js | 6 +-
heron/tools/ui/src/python/BUILD | 4 +-
heron/tools/ui/src/python/main.py | 86 +-
heron/uploaders/tests/java/BUILD | 2 +-
heronpy/api/bolt/base_bolt.py | 2 +-
heronpy/api/bolt/window_bolt.py | 2 +-
heronpy/api/component/component_spec.py | 39 +-
heronpy/api/metrics.py | 1 +
heronpy/api/spout/base_spout.py | 2 +-
heronpy/api/stream.py | 10 +-
heronpy/api/tests/python/component_unittest.py | 2 +-
heronpy/api/tests/python/metrics_unittest.py | 8 +-
heronpy/api/topology.py | 29 +-
heronpy/connectors/mock/arraylooper.py | 6 +-
heronpy/connectors/pulsar/pulsarspout.py | 35 +-
heronpy/connectors/pulsar/pulsarstreamlet.py | 4 +-
heronpy/connectors/textfiles/textfilesgenerator.py | 12 +-
heronpy/proto/BUILD | 4 +-
heronpy/streamlet/config.py | 4 +-
heronpy/streamlet/impl/consumebolt.py | 4 +-
heronpy/streamlet/impl/filterbolt.py | 4 +-
heronpy/streamlet/impl/flatmapbolt.py | 8 +-
heronpy/streamlet/impl/generatorspout.py | 4 +-
heronpy/streamlet/impl/joinbolt.py | 20 +-
heronpy/streamlet/impl/logbolt.py | 4 +-
heronpy/streamlet/impl/mapbolt.py | 4 +-
heronpy/streamlet/impl/reducebykeyandwindowbolt.py | 10 +-
heronpy/streamlet/impl/reducebywindowbolt.py | 8 +-
heronpy/streamlet/impl/repartitionbolt.py | 8 +-
heronpy/streamlet/impl/supplierspout.py | 4 +-
heronpy/streamlet/impl/transformbolt.py | 4 +-
heronpy/streamlet/impl/unionbolt.py | 4 +-
heronpy/streamlet/keyedwindow.py | 2 +-
heronpy/streamlet/resources.py | 2 +-
heronpy/streamlet/window.py | 2 +-
integration_test/README.md | 6 +-
integration_test/src/java/BUILD | 4 +-
integration_test/src/python/http_server/main.py | 14 +-
.../common/bolt/count_aggregator_bolt.py | 2 +-
.../common/bolt/word_count_bolt.py | 4 +-
.../integration_test/core/aggregator_bolt.py | 14 +-
.../integration_test/core/integration_test_bolt.py | 24 +-
.../core/integration_test_spout.py | 18 +-
.../python/integration_test/core/test_runner.py | 2 +-
.../integration_test/core/test_topology_builder.py | 4 +-
.../src/python/local_test_runner/README | 4 +-
.../src/python/local_test_runner/main.py | 2 +-
.../src/python/local_test_runner/test_kill_bolt.py | 2 +-
.../src/python/local_test_runner/test_scale_up.py | 5 +-
.../src/python/local_test_runner/test_template.py | 47 +-
integration_test/src/python/test_runner/main.py | 79 +-
.../src/python/topology_test_runner/main.py | 95 +-
maven_install.json | 5555 ++++++++++++++------
release/maven/heron-kafka.template.pom | 1 +
release/maven/heron-no-kryo.template.pom | 4 +
release/maven/heron-with-kryo.template.pom | 3 +-
release/maven/maven-pom-version.sh | 111 +-
scripts/applatix/build.sh | 129 -
scripts/applatix/ci.sh | 30 -
scripts/applatix/javatests.sh | 58 -
scripts/applatix/prepare.sh | 20 -
scripts/applatix/pythontests.sh | 43 -
scripts/applatix/test.sh | 89 -
scripts/applatix/testutils.sh | 46 -
scripts/ci/README.md | 4 +-
scripts/ci/build_docker_image.sh | 2 +-
scripts/ci/build_maven_artifacts.sh | 2 +-
scripts/ci/build_release_packages.sh | 2 +-
scripts/detect_os_type.sh | 33 -
scripts/get_all_heron_paths.sh | 4 +-
scripts/packages/BUILD | 4 +-
scripts/packages/heronpy/requirements.txt | 2 +-
scripts/release/docker-images | 2 +-
scripts/release_check/README.md | 2 +-
scripts/release_check/build.sh | 4 +-
scripts/release_check/build_docker.sh | 2 +-
scripts/release_check/full_release_check.sh | 2 +-
scripts/run_integration_test.sh | 5 +-
scripts/run_integration_topology_test.sh | 5 +-
scripts/shutils/common.sh | 28 -
scripts/shutils/save-logs.py | 10 +-
scripts/travis/build.sh | 37 +-
scripts/travis/k8s.sh | 4 +-
scripts/travis/test.sh | 11 +-
storm-compatibility/v0.10.2/src/java/BUILD | 1 +
storm-compatibility/v0.10.2/src/java/shade.conf | 5 +
storm-compatibility/v2.2.0/src/java/BUILD | 1 +
storm-compatibility/v2.2.0/src/java/shade.conf | 4 +
third_party/cppcheck/cppcheck-readdir-fix.patch | 12 -
third_party/gperftools/gperftools.BUILD | 20 +-
third_party/helm/BUILD | 2 +-
third_party/java/BUILD | 21 +-
third_party/java/logback.xml | 57 +
third_party/python/pylint/BUILD | 2 +-
third_party/python/semver/semver.py | 8 +-
third_party/zookeeper/BUILD | 10 +-
tools/bazel.rc | 52 +-
tools/docker/bazel.rc | 36 -
tools/java/src/org/apache/bazel/checkstyle/BUILD | 1 +
.../org/apache/bazel/checkstyle/CppCheckstyle.java | 16 +-
.../apache/bazel/checkstyle/JavaCheckstyle.java | 2 +-
.../apache/bazel/checkstyle/PythonCheckstyle.java | 2 +-
.../src/org/apache/bazel/cppcheck/CppCheck.java | 14 +-
tools/platform/BUILD | 36 +-
tools/rules/pex/BUILD | 2 +-
tools/rules/pex/wrapper/pex_wrapper.py | 6 +-
tools/travis/bazel.rc | 60 -
tools/travis/toolchain/BUILD | 60 -
tools/travis/toolchain/CROSSTOOL | 147 -
tools/travis/toolchain/cc_toolchain_config.bzl | 18 -
vagrant/init.sh | 62 -
vagrant/local-ci.sh | 4 +-
website2/docs/compiling-docker.md | 39 +-
website2/docs/compiling-linux.md | 64 +-
website2/docs/compiling-osx.md | 27 +-
website2/docs/compiling-overview.md | 35 +-
website2/docs/compiling-running-tests.md | 6 +-
website2/docs/getting-started-local-single-node.md | 6 +-
.../topology-development-topology-api-python.md | 2 +-
website2/website/package-lock.json | 78 +-
website2/website/pages/en/download.js | 52 +-
website2/website/scripts/Dockerfile.ubuntu18.04 | 1 -
website2/website/scripts/compile-docker.sh | 5 -
website2/website/scripts/javadocs.sh | 7 +-
website2/website/scripts/python-doc-gen.sh | 2 +-
.../getting-started-local-single-node.md | 2 +-
.../topology-development-topology-api-python.md | 2 +-
.../user-manuals-tracker-rest.md | 18 +-
.../topology-development-topology-api-python.md | 2 +-
.../getting-started-local-single-node.md | 6 +-
306 files changed, 6650 insertions(+), 4727 deletions(-)
rename docker/base/{Dockerfile.base.debian10 => Dockerfile.base.debian11} (97%)
copy docker/compile/{Dockerfile.centos8 => Dockerfile.centos7} (58%)
rename docker/compile/{Dockerfile.debian10 => Dockerfile.debian11} (90%)
rename docker/compile/{Dockerfile.centos8 => Dockerfile.rocky8} (87%)
copy docker/dist/{Dockerfile.dist.centos8 => Dockerfile.dist.centos7} (83%)
rename docker/dist/{Dockerfile.dist.debian10 => Dockerfile.dist.debian11} (98%)
rename docker/dist/{Dockerfile.dist.centos8 => Dockerfile.dist.rocky8} (96%)
copy docker/dist/{Dockerfile.dist.ubuntu20.04 => Dockerfile.dist.ubuntu22.04} (97%)
copy docker/test/{Dockerfile.centos8 => Dockerfile.centos7} (66%)
rename docker/test/{Dockerfile.centos8 => Dockerfile.rocky8} (91%)
copy eco/src/java/org/apache/heron/eco/definition/BeanDefinition.java => heron/api/src/java/org/apache/heron/api/utils/Slf4jUtils.java (71%)
delete mode 100755 scripts/applatix/build.sh
delete mode 100755 scripts/applatix/ci.sh
delete mode 100755 scripts/applatix/javatests.sh
delete mode 100755 scripts/applatix/prepare.sh
delete mode 100755 scripts/applatix/pythontests.sh
delete mode 100755 scripts/applatix/test.sh
delete mode 100755 scripts/applatix/testutils.sh
delete mode 100755 scripts/detect_os_type.sh
delete mode 100644 third_party/cppcheck/cppcheck-readdir-fix.patch
create mode 100644 third_party/java/logback.xml
delete mode 100644 tools/docker/bazel.rc
delete mode 100644 tools/travis/bazel.rc
delete mode 100644 tools/travis/toolchain/BUILD
delete mode 100644 tools/travis/toolchain/CROSSTOOL
delete mode 100644 tools/travis/toolchain/cc_toolchain_config.bzl