You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@qpid.apache.org by ch...@apache.org on 2020/12/10 18:53:22 UTC
[qpid-dispatch] branch master updated (06eef72 -> 31d69f1)
This is an automated email from the ASF dual-hosted git repository.
chug pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/qpid-dispatch.git.
from 06eef72 NO-JIRA: readme tweak, prod CI jobs
new dfe6643 Dataplane: Moved protocol-adapter functions from router_core.h into proto_adaptor.h
new 50ff2d4 Dataplane: Re-factored direct-AMQP to use the protocol-adaptor interface.
new 852b57e Dataplane: Exposed the protocol name with the connection entity.
new b5b4938 Dataplane: Improved sasl-plain test by using symbolic attribute names, not positional indices.
new 3bd7195 Dataplane: Added adaptor plugin capability. Started first reference adaptor (TCP).
new 2d241e5 Dataplane: Added documentation for the protocol adaptor callbacks.
new 1bf5083 Dataplane: Removed old handler call which is not used anymore.
new 513c365 Dataplane: minor cleanup
new 86cafb9 Dataplane: Added setter for dynamic in qdr_terminus_t
new a42f462 Dataplane: Renamed tcp_adaptor to reference_adaptor. Added more test content to the reference adaptor. It now sends messages to a fixed address. Fixed qdr_terminus_format to show the dynamically-assigned address for dynamis termini.
new d2b87bb Dataplane: connection-activate is now routed through the protocol adapter that handles the connection.
new ff4d6cc Dataplane: Updated the reference adaptor to implement connection activation
new 962d52e Dataplane: Fixed order problem in shutting down the router. Disabled the reference adaptor by default (uncomment the last line to re-enable). The reference adaptor causes test failures.
new c09b305 Dataplane: Added a 5th message compose variant to provide: - optional properties - optional application-properties - optional body in the form of a buffer list - indication of receive-complete
new b6982f6 Dataplane: Changed the new compose function to have only one field for headers. This field can have both properties and application properties. It's more efficient put together like this.
new 84f1d3e Dataplane: Added message method to set send-complete. Added reference code to receive messages (non streamed).
new ccd1b42 Dataplane: Exposed access to connection-ids from server. Moved the generation of the "connection opened" log from router_node.c to the core module. This causes the log to be raised for all protocol adaptors.
new 33066cb Dataplane: Added calls in message.h for streaming putput from adaptors. Renamed qdr_deliver_continue* to qdr_delivery_continue*
new a98fead Dataplane: Set proper buffer refcount in messages during buffer-extend. This ensures that the streaming buffers are properly freed when no longer needed.
new 8a2326d Dataplane: Updates to the message-extend (return buffer count for flow control). Added bidirectional streaming test to ref adaptor.
new b9a4a87 Dataplane: Added API for streaming data out of messages. This commit adds the requirement for Proton raw-connection support.
new 63ab97f Dataplane: Added no_route and initial_delivery on link-first-attach.
new a9368ea Dataplane: (from gsim) Implementation of qd_message_read_body.
new 3934871 Dataplane: Added implementation of qd_message_release_body.
new 18a1986 Dataplane - Added qd_buffer_list_append function to efficiently accumulate data in buffer lists.
new 2f4610b Dataplane: WIP changes
new 99f0aca Dataplane: Message parsing bug fixed: now properly handles empty var-length fields.
new 2c407b7 Dataplane: Fixed message parsing so it can handle partial and streaming content.
new cd70101 Dataplane: disabled reference adaptor
new 0cc281b Dataplane: WIP
new 4846f1c Dataplane - Added the body_data data structure for reading streaming messages. WIP - The following functions (in message.c) need to be implemented: find_last_buffer qd_message_body_data_iterator qd_message_body_data_buffer_count qd_message_body_data_buffers qd_message_body_data_release
new bcc6c80 DISPATCH-1742 Dataplane: Fixups from rebase to master.
new b85bed4 DISPATCH-1742: Added receive_complete parameter to qd_message_compose_3() and qd_message_compose_3() in message.c
new 3126244 DISPATCH-1743 - Introduce a HTTP/2 Adapter. This adaptor will act as the HTTP/2 <-->AMQP bridge
new 820b005 DISPATCH-1742 - Completed implementation of outbound streaming path
new 8244178 DISPATCH-1742 - Fixed compilation error
new f061cf7 DISPATCH-1744: refactor common HTTP code
new 365839e DISPATCH-1742 - Removed compiler warning by initializing a boolean value.
new 96e6021 DISPATCH-1742 - Handle zero-length body sections. Fixed a bug in body content length calculation.
new d533e9d DISPATCH-1744: Fixed minor error in field name
new 5aebe32 DISPATCH-1654: Initial TCP adaptor
new 70ea8c9 DISPATCH-1742: avoid using uninitialised value in conditional
new 4761a9b DISPATCH-1742: free protocol adaptor before core
new b398f8a DISPATCH-1654: initial tcp stats
new 0319443 DISPATCH-1744: Added libnghttp2-dev to travis.yml
new e947397 DISPATCH-1743 - Use new body data API to convert AMQP to HTTP and vice versa"
new 4dc9693 DISPATCH-1743 - Fix for leaking deliveries. Removed unused fields in http_adaptor.h and removed unused code
new 123a311 DISPATCH-1743: Freed all related objects on connection close
new 7cfa041 DISPATCH-1743: Moved the adaptor into http2 folder. Removed unused code and comments. Freed streams and connections.
new 634f1f4 DISPATCH-1743: Fixed crashes due to stream free. Still need to fix connection free issues
new 11442dc DISPATCH-1743: Fixed crash due to stream closure. Also fixed a couple of iterator and parse object leaks
new fe81acf DISPATCH-1743: qdr_http2_connection_t is now a pooled object
new d1a7e86 DISPATCH-1743: Code review fixes. Move write_buffers into send_callback() and remove all other write_buffers() calls
new 232de5c DISPATCH-1743: Accounted for length bytes in read_callback. Also added some log messages
new ce8d751 fix remote address
new a8c5825 DISPATCH-1742: Added new function qd_message_body_data_payload_length()
new aee0cb0 DISPATCH-1743: Copy the data only once in the snd_data_callback. Do not copy the data in the read_callback. Introduced qd_http2_buffers
new 76efced DISPATCH-1743: Added a no_body flag to the message content to know if the HTTP message has no body
new 625154c DISPATCH-1743: Fixed qd_compose leaks
new 085816b DISPATCH-1743: Fixed crash by not processing accepted deliveries
new 0524b29 DISPATCH-1743: Uses 16k buffers to send and receive data. Added code to respond with a 503 if server is unavailable
new ffd245f DISPATCH-1743: Added locking around connection freeing. Fixed body data leak
new a92c493 DISPATCH-1743: Some improvements around delivery settlement, fixed some shutdown leaks, fixed premature stream free crash
new fc170f3 Move deletion of AMQP adaptor to core thread
new 6d84ef0 DISPATCH-1744: HTTP1 protocol adaptor (WIP)
new eb226e6 DISPATCH-1744: tweak system test for python2
new 6f598fb DISPATCH-1743: Fixed variable names to http2. Also freed connector and listener objects on shutdown
new 0581939 DISPATCH-1788: fixes and testcases for HTTP/1.0 support
new 1f4b282 DISPATCH-1790: add read and write octet counters to HTTP/1.x codec
new 0de6341 DISPATCH-1743: Fixed nodejs issue and added code to handle bad client magic
new 303e4fb DISPATCH-1778: Extra data included in adaptor outbound streams
new f63b934 DISPATCH-1743: Fixed issue with two router HTTP2 requests
new 788ad5b avoid reading beyond end of body data buffer (#872)
new 5f6ee55 DISPATCH-1799: Add unit test for message body_data functions
new e1bb320 DISPATCH-1743: Added http2 system tests. Uses Quart to launch a server and curl as the client
new a08335a DISPATCH-1744: add multi-hop large msg tests
new cd414c4 NO-JIRA: fix RelWithDebInfo build issue
new 8502c9f NO-JIRA: fixup compiler issues after merge to master
new f394dc7 NO-JIRA: fix python lint error in http1 adaptor tests
new c0081c0 NO-JIRA: fixup entity count in qdmanage tests
new 9aff142 NO-JIRA: ignore async warning in http2_server.py
new d7a22ee DISPATCH-1743: Invoked nghttp2_session_send() right after passing the bytes to nghttp2 so that some callbacks are called that might have been missed
new 9c050bc NO-JIRA: Assert offset vs. body_data_buffer_count only when offset nonzero.
new 61165e1 DISPATCH-1802: HTTP/1.x prevent core connection activation race
new c497f5b DISPATCH-1743: Fixed crash caused by double free of stream_data object. Added additional log messages
new 0f71ade DISPATCH-1791: fix strdup and qdr_delivery_t leaks
new 579cbb0 DISPATCH-1743: Fixed format specifiers in log messages for connection and stream identifiers
new 21d97a1 NO-JIRA install nghttp2 in Travis and GHA CI (#886)
new b1f7312 NO-JIRA: cleanup field leaks and format specifiers.
new f2d1f16 DISPATCH-1744: fix input body parsing to avoid inserting empty buffers
new 2f6498f DISPATCH-1744: fix cleanup of outstanding requests
new 729c41c NO-JIRA: more coverity error cleanups
new c8457b0 DISPATCH-1803: prevent body_data sections from violating Q2 limit
new 196c1be DISPATCH-1807: TCP adaptor test echo server
new b9ce765 DISPATCH-1654: need to set to field on message
new 9ead53e DISPATCH-1654: allow content to be seen when available
new 88ff655 DISPATCH-1654: fix for streaming message
new e395310 DISPATCH-1806: Rearrange TCP adaptor outbound body data handling
new ee36b88 DISPATCH-1807: TCP self tests - add a TCP_echo_client
new 7198e3d DISPATCH-1811: Check credit before sending delivery. This fix ensures that the edge router case works. Also added system tests
new b69555d DISPATCH-1812: Added the preix C to log messages that did not have them. This makes things uniform across server.c and others. This closes #889
new 41aac50 DISPATCH-1779: initial http stats for http1 and http2 adaptors
new 43936de DISPATCH-1768: Renamed qd_http_listener_t to qd_lws_listener_t. This closes #842
new d436b3f DISPATCH-1807: TCP self test - improve test server
new a44e7ad DISPATCH-1807: Replace assert with raise; fixes tox test failure
new 7bdde08 DISPATCH-1813: Renamed qd_http_lsnr to qd_http_listener. This closes #894.
new 9f70c1f DISPATCH-1815: Send PING frame every 4 seconds on an egress connection. This closes #897.
new fd1171f NO-JIRA: add dummy system_tests_tcp_adaptor.py for temporary CI fix
new db8581e Revert "NO-JIRA: add dummy system_tests_tcp_adaptor.py for temporary CI fix"
new b47762f DISPATCH-1807: Add TCP protocol adaptor tests
new 0b4f4f6 DISPATCH-1742 - Added function for asynchronous completion of sent-messages. Improved API docs. DISPATCH-1742 - Updated reference adaptor to make multiple extend calls. DISPATCH-1804 - Changed body_data API to stream_data to encompass both body_data and footer performatives.
new 7063d58 DISPATCH-1816: prevent activation from running after reconnecting DISPATCH-1791: avoid leaking qdr_delivery_t
new 5c99f50 DISPATCH-1784: allow parsing of folded headers
new d55885f DISPATCH-1812 - Fixed test that fails due to log format changes.
new 107003c DISPATCH-1742 - Fixed leak of stream_data object in reference adaptor.
new 7af722d DISPATCH-1807: Add self tests for tcp protocol adaptor
new 708a4a9 DISPATCH-1779: Additional fix to use stream_data->request_status on the client and server side
new 2962ead DISPATCH-1742 - Remove buffers when refcount goes to zero on send.
new 34f6274 DISPATCH-1776: Added GRPC support using new body footer API. This closes #908.
new 58025a5 DISPATCH-1817: TCP adaptor leaks streaming_data_t objects at shutdown
new d10a7a6 DISPATCH-1823: TCP adaptor honors discard flag in core callbacks
new 487ebe8 DISPATCH-1820: TCP tests hang, this gets many of them started
new fc44d80 DISPATCH-1824: Fix TCP adaptor listener and connector shutdown leaks
new 4ed3af6 DISPATCH-1806: Account properly for write buffer byte count
new d8f75a9 DISPATCH-1820: Improve logging to investigate TCP test hangs
new f274d25 DISPATCH-1820: Fix python tox errors
new e4acadc DISPATCH-1825: Skip TCP self test if selectors module is absent
new 370716f DISPATCH-1654: fix to ensure connections on egress are closed in all cases (patch from Ted Ross)
new 9e35f25 DISPATCH-1826 - Fixes for: - Accumulated temporary address records - Race condition where ingress message is sent before reply-to is established - Deliveries, messages, and buffers are leaked when connections close
new 61a26b6 DISPATCH-1776: Check for presence of next_stream_data before trying to free it
new 463df41 DISPATCH-1807: Rewrite TCP adaptor self test
new 3d6ac30 DISPACH-1807: TCP self test integration with scraper
new c337362 DISPATCH-1776: Additional fix. Create nghttp2 client session on creation of new connection
new 1fdc2f0 DISPATCH-1807: Improve TCP Adaptor self test
new e501df9 DISPATCH-1830: TCP test allows concurrent echo session
new dc1dd7e DISPATCH-1743: Added some additional log messages
new 60a57ad DISPATCH-1830: Fix TCP test glitches
new cba4d3e DISPATCH-1743: Prevent qdr_http_flow from calling route_delivery() if in_dlv already created
new 7c93e09 DISPATCH-1832: ensure correct signature for deletion functions to avoid mangling of pointer
new c985b83 DISPATCH-1830: fix inadvertent file rename that hides ECHO_CLIENT logs from Scraper
new dea6365 DISPATCH_1829 - Patch from Gordon Sim
new fce8333 DISPATCH-1826 - Drop raw connections when stream messages are settled.
new fd5ac38 DISPATCH-1654 - Removed the redundant copy of the ma_stream flag, leaving it in the message content only.
new 87b7134 DISPATCH-1776: Set the qd_message_set_stream_annotation() flag on all messages
new 16fabe5 DISPATCH-1840: avoid race between adaptor and core cleanup
new 1d85618 move response disposition update out of link_delivery callback
new b64fe0b DISPATCH-1844: suppress leaks of sys_mutex (shutdown leak)
new 57751ed DISPATCH-1826: Fix tcp adaptor stall on large messages
new b29c5ea DISPATCH-1831: Extend TCP adaptor test interior router backbone
new bd650e3 DISPATCH-1842: Handle managament deletes of httpConnector by deleting connection objects associated with the connector. This prevents the router crash from happening
new 3b9be5c DISPATCH-1846: Improve tcp adaptor basic connectivity test
new a575c35 DISPATCH-1843: disambiguate http stats records
new b3f36be DISPATCH-1847: use remote host in stats for ingress
new 81e1b15 DISPATCH-1848: TCP echo client/server handle socket errors better
new 7319457 DISPATCH-1849: Don't dereference a peer delivery that has already been released. Check the in_dlv_released flag before dereferencing the in_dlv. This closes #925.
new 035adfe DISPATCH-1841: HTTP1 remove connection when connector deleted
new 4e1c7e9 DISPATCH-1816: avoid race between conn disconnect and activation
new 12cd66a DISPATCH-1845: cleanup input buffers on server disconnect
new 9b96d79 DISPATCH-1852: Added code to accumulate DATA frames in the message body in case credit does not arrive and the header has not been routed. This closes #926
new 49f61d4 DISPATCH-1851: TCP adaptor - add more connection and link IDs to log output
new 2b43538 NO-JIRA: remove unused octet counter
new c88f0c7 DISPATCH-1849: Additional fix. Changed name of flag from in_dlv_released to stream_force_closed. Close server stream if delivery released on server side. Also added code to properly incref and decref qdr_deliveries
new a2cc13c DISPATCH-1851: Show proper link id in second_attach
new b62f706 DISPATCH-1856: avoid adding empty message buffer
new fd84a4c DISPATCH-1833: fix buffer fanout decref when releasing stream_data
new a2a2b3d DISPATCH-1819: HTTP1 fixes for concurrent streaming test
new 4cb9ed5 NO-JIRA: remove stray proton library function definition
new 24b015b DISPATCH-1857 - Handle asynchronous moving of a delivery to another link. Patch contents from Gordon Sim. This improves the stability of multiple, concurrent connection handling. This closes #932.
new b69bfab DISPATCH-1862: TCP adaptor - rename structure members
new 2e3bab3 DISPATCH-1863: add proton event names to event handler logs
new 6d5908a DISPATCH-1744: remove old stream_data parse bug workaround
new ef2f270 DISPATCH-1846: Fix TCP adaptor test stall
new d3ca291 DISPATCH-1872 - Fixed handling of closed server connections in the TCP adaptor.
new ee5797f NO-JIRA: Tcp test client data verification success case efficiency gain
new 514d03b DISPATCH-1868: Correctly used stream_data->body to accumulate body data segments in the case where the entire body (DATA) of the HTTP2 message arrives before a delivery is created. A delivery is not created in cases where there is lack of credit or an absence of a reply-to
new ae64db7 DISPATCH-1873: TCP echo test client prints errors to stderr
new bf81c3f DISPATCH-1874: DISPATCH-1875: Free the unused buffers on a stream that has been force closed. Also free unused compose objects and pn_data_t objects
new 6cd2802 DISPATCH-1874 - Additional fix, removed bad code
new b7be656 DISPATCH-1875: Fixed pn_data_t leak
new 54c2079 DISPATCH-1875: fix pn_data leak in HTTP1 and reference adaptors
new 442062b DISPATCH-1877: TCP echo client test has connect/disconnect capability
new 9c31d2e DISPATCH-1876: reproducer self test
new 4caac05 DISPATCH-1869: discard invalid messages arriving from core DISPATCH-1859: re-work server connection cleanup
new d8a3598 DISPATCH-1857: locking around activation
new 2d5bb0e DISPATCH-1876 - Added proper ref-count protection for the initial-delivery. Fixed leak of buffers when connections close very early. Enabled the connect-disconnect test.
new 6de515f DISPATCH-1779: fix use-after free issue flagged by coverity
new 47e35ae DISPATCH-1867: Initialize modules and adaptors outside of the core thread so they are ready to go right after the qd_router_setup_late() completes
new 31d69f1 dev-adaptors merge: comment out stray decorator
The 188 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/workflows/build.yaml | 2 +-
.travis.yml | 5 +-
CMakeLists.txt | 1 +
LICENSE | 4 +
README | 12 +
.../Findlibnghttp2.cmake | 18 +-
dockerfiles/Dockerfile-fedora | 9 +-
dockerfiles/Dockerfile-ubuntu | 7 +-
etc/qdrouterd.conf | 1 +
include/qpid/dispatch/amqp.h | 2 +
include/qpid/dispatch/buffer.h | 25 +-
include/qpid/dispatch/http1_codec.h | 245 ++
include/qpid/dispatch/message.h | 157 +-
include/qpid/dispatch/parse.h | 2 +
.../dispatch/{router_core.h => protocol_adaptor.h} | 645 ++---
include/qpid/dispatch/router_core.h | 648 +----
python/qpid_dispatch/management/qdrouter.json | 250 ++
python/qpid_dispatch_internal/dispatch.py | 9 +-
python/qpid_dispatch_internal/management/agent.py | 55 +
python/qpid_dispatch_internal/management/config.py | 2 +-
src/CMakeLists.txt | 13 +
src/adaptors/adaptor_utils.c | 147 ++
src/adaptors/adaptor_utils.h | 54 +
src/adaptors/http1/http1_adaptor.c | 715 ++++++
src/adaptors/http1/http1_client.c | 1344 +++++++++++
src/adaptors/http1/http1_codec.c | 1658 +++++++++++++
src/adaptors/http1/http1_private.h | 275 +++
src/adaptors/http1/http1_request_info.c | 46 +
src/adaptors/http1/http1_server.c | 1533 ++++++++++++
src/adaptors/http2/http2_adaptor.c | 2533 ++++++++++++++++++++
src/adaptors/http2/http2_adaptor.h | 212 ++
src/adaptors/http_common.c | 690 ++++++
src/adaptors/http_common.h | 126 +
src/adaptors/reference_adaptor.c | 588 +++++
src/adaptors/tcp_adaptor.c | 1529 ++++++++++++
src/adaptors/tcp_adaptor.h | 90 +
src/amqp.c | 1 +
src/buffer.c | 34 +
src/connection_manager.c | 2 +-
src/http-libwebsockets.c | 40 +-
src/http-none.c | 4 +-
src/http.h | 6 +-
src/message.c | 926 +++++--
src/message_private.h | 54 +-
src/parse.c | 12 +-
src/python_embedded.c | 2 +-
src/router_core/agent.c | 18 +
src/router_core/agent_connection.c | 54 +-
src/router_core/agent_connection.h | 2 +-
src/router_core/connections.c | 243 +-
src/router_core/core_client_api.c | 8 +-
src/router_core/delivery.c | 20 +-
src/router_core/delivery.h | 4 +-
src/router_core/management_agent.c | 30 +-
.../address_lookup_server/address_lookup_server.c | 2 +-
.../edge_addr_tracking/edge_addr_tracking.c | 2 +-
src/router_core/modules/mobile_sync/mobile.c | 6 +-
.../streaming_link_scrubber.c | 2 +-
.../modules/test_hooks/core_test_hooks.c | 2 +-
src/router_core/router_core.c | 112 +
src/router_core/router_core_private.h | 67 +-
src/router_core/router_core_thread.c | 62 +-
src/router_core/terminus.c | 18 +-
src/router_core/transfer.c | 106 +-
src/router_node.c | 137 +-
src/server.c | 90 +-
src/server_private.h | 14 +-
tests/CMakeLists.txt | 5 +
tests/TCP_echo_client.py | 326 +++
tests/TCP_echo_server.py | 326 +++
tests/buffer_test.c | 35 +
tests/http2_server.py | 90 +
tests/images/apache.jpg | Bin 0 -> 5147 bytes
{docs/books => tests}/images/balanced-routing.png | Bin
tests/lsan.supp | 3 +
tests/message_test.c | 616 ++++-
tests/quart/LICENSE | 22 +
tests/system_test.py | 48 +-
tests/system_tests_http1_adaptor.py | 1692 +++++++++++++
tests/system_tests_http2.py | 640 +++++
tests/system_tests_qdmanage.py | 2 +-
tests/system_tests_sasl_plain.py | 37 +-
tests/system_tests_tcp_adaptor.py | 704 ++++++
tests/system_tests_topology_disposition.py.in | 4 +-
tools/qdstat.in | 4 +-
85 files changed, 18816 insertions(+), 1470 deletions(-)
copy python/install_python.cmake => cmake/Findlibnghttp2.cmake (70%)
create mode 100644 include/qpid/dispatch/http1_codec.h
copy include/qpid/dispatch/{router_core.h => protocol_adaptor.h} (58%)
create mode 100644 src/adaptors/adaptor_utils.c
create mode 100644 src/adaptors/adaptor_utils.h
create mode 100644 src/adaptors/http1/http1_adaptor.c
create mode 100644 src/adaptors/http1/http1_client.c
create mode 100644 src/adaptors/http1/http1_codec.c
create mode 100644 src/adaptors/http1/http1_private.h
create mode 100644 src/adaptors/http1/http1_request_info.c
create mode 100644 src/adaptors/http1/http1_server.c
create mode 100644 src/adaptors/http2/http2_adaptor.c
create mode 100644 src/adaptors/http2/http2_adaptor.h
create mode 100644 src/adaptors/http_common.c
create mode 100644 src/adaptors/http_common.h
create mode 100644 src/adaptors/reference_adaptor.c
create mode 100644 src/adaptors/tcp_adaptor.c
create mode 100644 src/adaptors/tcp_adaptor.h
create mode 100755 tests/TCP_echo_client.py
create mode 100755 tests/TCP_echo_server.py
create mode 100644 tests/http2_server.py
create mode 100644 tests/images/apache.jpg
copy {docs/books => tests}/images/balanced-routing.png (100%)
create mode 100644 tests/quart/LICENSE
create mode 100644 tests/system_tests_http1_adaptor.py
create mode 100644 tests/system_tests_http2.py
create mode 100644 tests/system_tests_tcp_adaptor.py
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@qpid.apache.org
For additional commands, e-mail: commits-help@qpid.apache.org