You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Tom (JIRA)" <ji...@apache.org> on 2017/11/06 19:28:00 UTC
[jira] [Updated] (SPARK-22459) EdgeDirection "Either" Does Not Considerate Real "Either" Direction

     [ https://issues.apache.org/jira/browse/SPARK-22459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tom updated SPARK-22459:
------------------------
    Description: 
When running functions involving  [EdgeDirection|https://github.com/apache/spark/blob/master/graphx/src/main/scala/org/apache/spark/graphx/EdgeDirection.scala], the _EdgeDirection.Either_ type does not truly perform the actual specified direction (_either_).
For instance, when using _Pregel API_ with a simple graph as shown:
!http://www.icodeguru.com/CPP/10book/books/book9/images/fig6_1.gif!
With _EdgeDirection.Either_ you would guess that vertex #3 will send a message (when activated) to vertices 1, 2, and 4, but in reality it does not.
This might be bypassed, but in an expansive and ineffective way. 
Tested with 2.2.0, 2.1.1 and 2.1.2 spark versions.

  was:
When running functions involving  [EdgeDirection|https://github.com/apache/spark/blob/master/graphx/src/main/scala/org/apache/spark/graphx/EdgeDirection.scala], the _EdgeDirection.Either_ type does not truly perform the actual specified direction (_either_).
For instance, when using _Pregel API_ with a simple graph as shown:
!data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAPkAAADLCAMAAACbI8UEAAAAh1BMVEX////9/f0AAAB1dXWFhYX4+Pjl5eV6enr39/fj4+Py8vLo6OgsLCyCgoLs7Ox8fHxTU1PKyso8PDxNTU0vLy/c3NwiIiJFRUVLS0vFxcVcXFzOzs41NTVSUlIYGBisrKy9vb1tbW1mZmYeHh6fn5+UlJQLCwuysrKMjIylpaWZmZlgYGASEhKECRTxAAAUhElEQVR4nN1diZqqOgy2LIq1ICAtssi+Ku//fBdwOY4UBEThu//MOAoUG5ImaZq2q9X/FwAIo8rxYOKK/BBA2KZR7AeBL8a5Bfs/AMHI83jv+34W5XmOv1jFr8CK96aalGCY8pdB+mUTYf5tMWBFG9kpCzBhVYxhCtXPrB/UdyIAHNlJeAxKpsXnPI7OMcseCSGbdwzcnu2kUE/+XsyyjNv7sh4miaOeezyzRQDHqqNm7hqW72/NledxepaJHnUJPYxUpAYxhpAHJcrCa2zkrJ4kdrb7RcU/BHCDwuaa3AWrXaSFm3VbOd4wic1ZTc2GMzV0ZMqJhcHyyYXDkH7SuxDZo0vuWtSci0EVCR6zqBT5hUs8NgvVa2fPLkMoppGAzdAWW0UaGAdCsnH28UfwdOQrXRfwkYYoJKx9JHstglLD2iCUflq7L8LwL7nUfQmfayh+JX3ro3d6H57Rwfioct8E9IP8bWsUYqJFfw9tfT1/K8qCSI5LteyYVdMeagjGmvqHBLi3ox6aW8hQ0NUgZkSE2F41gxninlns2cGbJnKFFGj5Im0b1o89ney1bz9pK3xQvX7lPKQusanDPXL7cgQf9cdDgpzq9jTVvIhOC+zBuMjvJbM1zuRh1Y1T/9a7NcPo/VU/BjTJAHur2ObNbeHPh96iUl5sn7ZDa/ZtpOGlheVw2+SpsFfd6ztFN1td+SZ2pu0Nr9tXAQO6iwWMWFbZpq+OD1emg4hklFbOt5h3/kz8hVk2bNNZt/aD4ICaypv3UX1sLasU92SdtTVnrB4WJu4GEmkKGvA7QVhz6NJgVK7l1b9UE5vs3fmkVetlaFk6DkSow9JuZb1hjPBRrP7FtEYice2Up4RdVJ8NmqjDpwbnpkivTbPUiPzmKvR/wXtEbuuxKkTubzx/gPVB7Wp+udrgOfQrOYCmTZMVy242j/spTZ+3oe/+1qxUPB0qV4gODcr5fSUH2yPV/VkfWxkL2a529X1g3WYNvH60OAt1UQ7yJn1Cpipliz5SFaNkHlqNfMyYhrCarefiVrFwZG9S7zqEYGmdrS81GjXlxar3IR1EGg3wcmqNTJ0ZJrlkMcuyG/YZm9cDj8N/P2wopxp3aj27sZkbCt2snDGsyl1coJwrex9eSaKc0a6Hl/YHGdfjEEdmbpDDvmrCW72TctrDyEjFc5Nqo2BwbEr79QuEDcPYMptxIseJ1R93f7m+eTpw/aUcq47e3v09frv8dlb8c1a8nj3UZGuXzChVHag01WGglRX2VWwJBjrNGkJfbmvnGDEBXkPIl/pBACu+RPkCeFB6Tav6FQig/OUBqE+Uf7dj5Uv9Qag+VB9X1VX3O6yuZerj5UtdqDoL/p4FHsMc2HwNbrwAWFcHjoJIgV8WFzYUTQ3KB9LW/VlhMq9uN3TxblVr6iEbDgwZbI/7siSfUfu2MDg8Pcg/7chCTd/g23iugIBfhDstWjgBbs3zVQ2khKv+xdRQAwxaNZxHKCpgTlgtvUclOOMS6eWlOQPROVf/PZsWtdwdWx211Nkvym8vFQ/VDYU+cTQb2Yh7oY9n664KWJvNzsyKzwly6fSVxnBhQXeeZagxcyvecGKU5a+MtdRbbyynxC2lgJC6uw9ePQGgkJnd9iYsctnRPJbKuDRR9tFumk3RmiENYLmpQY3Hlpp0aTGZFbiQPgMlNyja3WDDPXX4ANDvZenURjUvoqI/N0AcPobDPdQ/BAk5euhnXuCg/0iDYf8zyoKI4r6PzFCHhLZ/BsvWO4fOn640n4cQLV3rSc72tLjIaw0+dsxegSLoh+cn6RBiTe71yCDnyAszaTdgk8Q9SIcx+su58kn0GRkXIqK2JNnMDmweuLcOFh8T9YXFyom0jj8/hEPgQtJneP7HgEZaddks9q22AoZ2bBgmQ9eydwreUENOmnX4vPHlQLI8Vq/HDVbYJ2KnwPPuEVHst3VyLp2JjoKrE263rLwBKQ9QUsS3SmGZ+B3upZQRmzpabhyKY8PF/Qfo2uZmvSDCBSvb6AzDJP7Vy15VyTLEj1vYBz0ZXVrSBPCGELEl0REoAdIWlQAKYnKNxvn4X5UFN2DUPSV6wBuXkrjW+kNXLTS/EaQFAtjFKLGjRY2srLxrKjb5Y5OAFZ0cPVKe1REPjXyDSNCZ7af4JNQy5Q+J/DaND6EmGssifIVrnheNJLh1bDPMSTy7blr+pjkXHAsGsdYbvS94HGIcOXY9pYLhpRHHOqVIzTm0QIXg1SynjP7xZcv8E50OD5tOft8ADFO7ytH1lUk2F/MsrZYVhlntxLp69K4WxO7eN+WDXvIsMSOjb39MslL2UhYq6lEM3zXqPPllcRxuSlZqTOi2XQAEWGKXE0rAqfPGcGuphFHT7U6CdYb/HR/XeRJg0XE2FnbfZqxaqHXYoB0RKaKa5H9kj6d80icmGDqj57DPPS1tRJzYu/bgwV8Mvs30kDIUXpoDpFRYqH1IuBWGXUfcF0f5LitI70CKMkbaFa1+XAujHFiyc/R6X25pY3iu1rZyWZTDsx129UpeUWq44S63ZR8WRzk8O2g/hJRRlJftfGHSDhSTmMNiQsoYDacsTcMJoubsB3YeRtnzpWk4KQvVd5OTGsDaYYS0XzXcaiGUl8ZsRJL1JxruhfTBt5kGA43ZA6UnM3z486bhFkE5FAmiTLR9j5Ea7nRtVfNTbm2QPi7Ob2mnMRrutAyeC6X3Eowc2flIw80NKS69l7EpSRYyhxPxkPZ5gc1Q9UYHhJQxGq6U9vl5DoyDo3njy4/rq/2K5x26A0QobIbBBwCPiUwY9uw8FyKinT8arh/lyczfzrcbcvwwQWOUPS89mVl5zhtmr0H9TljaqHY+whROB5jbIftxfu2oOTe3ONwTfunHbFnSXBJiOJQBaV8PWGPcn6mA/eLgTpCKhO1Run22aWm8qxbBJJnko6LO3e38m4IPzwRl0zz1UZ7MbDyXfMd+v9BJP1hjpF2hh6q/reUEz3T0jjWQhmHcGMssfTUpQoS1Jks9GyXt1hz2fMcmagSpaemjMEra52jnilz0WgqoN0ppHxWZ+DHlIFVD/10+yzCMk/ZfezIwVd+u2jUUo8bPDfVXy6fUipzHG3JKp84gHyXtnjoibHnDcKPEe6oTvE7B+xzjPBn9h+18d7Yd/wszH0eNnyv67xYLMoLCPn9j4qPVI5jYEFDvR7odrIBnJ+oUPbMmjDE+nHUc384HYReHZOxIwjuM6p+XPP8J5copIe63JgyMs2r6L/x2wZMZdaqeWRMKGiG4P6EcRkXyzcWDR40o/oJyzIZkj7/Y8S2lfUQEsocP91mdBUVm7I4JJBNgrIb7rlWDqZ0cvrxMtqXpy7Nq24wknwfU32CchvvuCnjrS4LEr5vNUd5rP2kfJ6w8NExG9hqLfkyOURruQ2nvJApGhOk3B/hDjOurvUj7hPzZZU6x/42DiEbw/Fu6HfCWX6Bs95Np7OOiUV/S7YKnM+rkwZcWjMyTGaEW3wEAHCHmd5PYR42ffycOt2WTxKRvg/ENjBo/974xxmKYBcl+uKvNKO9V0SfPEAKG5qg/ndg7yp4rqjyxtEsxSnqNmU1nQJfhvW43ReJ/s0tKwSjKp7bnipyQ+NcLy42i3NKntOdCZCdy+vOZvePmq9kTtvNtFibBDFtWKWicVZuK57xyKtB5jgVZRkYgp7JqwNAT9f3WQN/AKMonG10CLnHMH+v0O0b1WKaSduBp4evqmz/DuIzfaaLOOEDkPFvS9CSRiVEA2Eyai1T9DrNFJoRUJ7922/5gXGTic09GEp0wm3WJ1HFZYR/3WCzfUXuvi/AdGCOl/SMNBzzdORozLxo5R/9cYkkydY7XcIzScJ95MjBznPY9aX+G33sy601hD54m/wWMauetnsx7I8XjYzFmtvj0GEX5q7T3t8ogr7L6Bn/hN1BSPkraOzVc65OArk1YDDqv+RWUwdmffO7bSRFw3uDvApbPaN9JbqN+XddJz9SZRBs2zxHK9VKDTNayIGZ7TVK76L8+9peRMtcFE4eIHu/XhQbvS8C7tn7x5p7Je8darRdD9IaUAW5RFaJtsEi7+v4G5khP57fid8CafwM3NMbVIpKFN+ybsIj0F9H6uo7r/AKjopwbVgfIDhZ2mKrkG5naFPRd82NbLYE5dCfnvFpZcwgdkCOh+aOoU1/K4YlhaDuDdsJDvZt5DYlDRPyV+9J7nZeSf4P72pV2GBAp3sYEZb+z4n0pt0Jm+O4K54L0niQJDTk8RL+z4r0px2oyfL2GFPUOkksZKQ7KD1cB7k059EfsiIYPrfsXvMDahyH700Gz/us5uSO2a98Fm17NnFfk5Oh+2BcH49F9Y2kER2Cc9ikk5GooG/yHHP8a5eNqI/RYQHgdoZD7PKK+HMrrbRwhfivDuNoEYILdG75AueR5abV4fe7B/sYZSJaRRucgvMRn18DtoozNYpqJZh9QTnkMYOtmwWM/1+J4EdNe1EPjHJB/K94X6ubsrWk9A8E7Fo+lpT/j+3SUAx56rM7Yuslu2GzP7S9HFTHkwCrvOASwX3ZQQ9vWCFFl82iHpdOvmxGGr6yHKZlo7ZMpKZfik5OomYWrLVlXvACgtFM43Sns7roK+cVhCjnaStIO450EYSn4sV6z/mUGwu5sOv5U6YyfUfsPvMWGoe81QgRrSySJ3rGcwTYOi8J3d9crHvfELpuUtFdbZt5lW7Ay38wm25JnIsphfmGOdMUjKH6itnaprBMTBqlEabS8wZVdXfMx/4J3j0d2wjjjh5TfaiVljNm+3YyU2YVPPcu7NtPe34LVNieHm+WGqY18o9Hyx+Mjyu9yiGVGp297eYNydGhTf4GLwmN7/jWAxoVhqv3jAcB7gvZYmHDxsCkot+RCfbNyoqE6zbVAhVxDYnf4Ap6LZIN5wTgm4cQZyx9RfsVuj9h3k4JAqpLshbl8jt5PsdiWim5jxBpTTLSg1VOdPiG8+oOczL2vFPB08jcux3t2n5WysXkNwy9hmPQFbs/op6f+Dcxhk7bbXBOKU4Xue3XifghQ7yrXc/GhCJlPkbltQM69ikli6dF9thDpV8CLTvdOgv8gsWH8j3Ne3+2J+C3LkLce8K8BVgZCvRugZf+Td4l1niKUb6KZMvO6n/j8kHxn3yrr/MsZwQ/vF/Nuqdcp9PIC5W7wzGi/mFE6CClpz2zB4msM0rNvm6aC7YVEDQ+Gh4boZ41oC+AVNYkWkBPwDGi2j5euffR6js/ItaULETlYDXfMuviBGh4oTlFKgqUME18BLGS3qqncCRsRd8O+pvRCn7aD087bQYUlm6biWB+I91FNpwYQE7YtZLj1SXP7ubV53Qgcmm2KHVg6ZVQK+sV5UcIuXFpZAV2cOXnjcGTX2t2yW/dy38lmM7wIcrJfhna/d06Pbc2PtyzedZpbDnqojjXEpPlQbtiaNM8OqwM3J/wyUmdD5wSfp2UX1GnSYNkXqaLj0EYHMGJa+1kf7L+GQphX48VMRG990AMr/tzUcLdlYA27jYOCF0Q0kw6yQnx8FYA7hfXHVXkixIxLpRxUOgq4YfO5rOWK5ruKbxb01NIZpkUfooLlK9O+xkaa6ShhyJwaD5htlNevadFMbRPEKinUQy08BxZ3cKhDkW4SSKXZi7V7WD7ZbPblT/1yfbO/vd3/e/vv9J+f/f26fwU21+P7x83uhTfNouXBgqGOkeNU4HkAI+bS8MdgVpk1ry0pFpT+ml3Q+nBRlYgg5RdmKaBRzu9tNsu4DWKS46uOA2JFuaW1a2o+J5Q9bED5GKWygVeR+DptjklEcc+JHMttqhex/uGqV/H+Vx3YPA5X//f727vbz/PHxz3+fKjei4/D17tWfwUTU6QdiLqqHg+lWCaH19awliuDju2ORGiLpvh5jjHrSCQQ1unmmDCMDer1HMF1W9lVHRS8hQavf48T9YfVv8vAv8/Pb+/3eLrufgY8bnG7ednORVrgFErrNYRWwJgN+q404zYNV19Cc+JAXPH8fn/sHplL6w1+gRh1ZXfklBwh61QqqsqJbQ+5KirFVPIiEz8ZO7CNWl2hn8AouraVSkOuYZkNVKeExoTihgt1fxd4tB1RtvqLJzPzZqJbvSsNaGs2meeptWZwCaWZbA+RBa3sTItEY/2vlICZKS+7UF7HaavZXqNrXutWpyTLWWpoy2a+pnmvORKX0WO5AXDM+Wl07T3Wp6s5E3zUtIeClcUGPf+FZ1GLnzwXPGfYmgqWbdas49kwopxuFWHJXFogDurDYiW5euu4elrT4l1BJd6z2aUNsbjakIgBPtw9GHgZsm2ddCHNrv7M2PmH/rncIHsMNQgZ6jEYd4eCWvvzs6HsVrYEJyiwbP3RWvFR6x1ZkzjkzrJMSCcAh/qmpAOOnB+X8i7S+6YE5yhYzhSdf8BHtZ/aBantPwm4FJCeO8mnuuqNqNjXAc7E79PU+fSla+qpqJd29GyNHv+YHWvxKCvvBd6zX+KtMEdOj4ayDpy/7tvMfusTwG5fvJ8wvzbDl3QPsItJ8i6hEex857I4vX4Dv8KyI3cLPPD08LVVA7ANwmTTHT2WRGfQPK5fw9KZY1f9SkWe0FJKcMAwlw79CDyfHOZcG+YteCNw9u2TQ/EmDOm5NDgIGXRuow1HqiN35tnNDmEFoyBRc2pr3521amk+OnW76MAUukuhXbBi29HERXO8huCpTmKet88BBwAAxOdjQqql+doo2PqOk2iiB/8lwgNBWJ9ZjUlOqbBqL/k9vMsA/AsB57LDqJfIwjtJgnC9W+PUP6HEZo1OD11KTSdhHJPNPLwrsTayQLYZhrG57UwMH0J5HZ8oaUiqDH1dP8ny8aRWCfrq/m0+OpDcU1KH0DVbVdUqr7/E8Yxni8IM43mNbZQFJLmNQiRoH7lKnzFPCbu+7jyGLxI1yPIfrkfcwAjKyya6tdLU8/Iz57qKBPpGqgDc5bF7jsWA5bIIS/Mq9DGU/yv7Uvf3hUClHIUdzw+Y8fQlfEJ5F/4D+Bt8+EFHu9EAAAAASUVORK5CYII=!
With _EdgeDirection.Either_ you would guess that vertex #3 will send a message (when activated) to vertices 1, 2, and 4, but in reality it does not.
This might be bypassed, but in an expansive and ineffective way. 
Tested with 2.2.0, 2.1.1 and 2.1.2 spark versions.

        Summary: EdgeDirection "Either" Does Not Considerate Real "Either" Direction  (was: EdgeDirection "Either" Does Not Considerate real "either" direction)

> EdgeDirection "Either" Does Not Considerate Real "Either" Direction
> -------------------------------------------------------------------
>
>                 Key: SPARK-22459
>                 URL: https://issues.apache.org/jira/browse/SPARK-22459
>             Project: Spark
>          Issue Type: Bug
>          Components: GraphX
>    Affects Versions: 2.1.1, 2.1.2, 2.2.0
>         Environment: Windows 7. Yarn cluster / client mode. Tested with 2.2.0, 2.1.1 and 2.1.2 spark versions.
>            Reporter: Tom
>
> When running functions involving  [EdgeDirection|https://github.com/apache/spark/blob/master/graphx/src/main/scala/org/apache/spark/graphx/EdgeDirection.scala], the _EdgeDirection.Either_ type does not truly perform the actual specified direction (_either_).
> For instance, when using _Pregel API_ with a simple graph as shown:
> !http://www.icodeguru.com/CPP/10book/books/book9/images/fig6_1.gif!
> With _EdgeDirection.Either_ you would guess that vertex #3 will send a message (when activated) to vertices 1, 2, and 4, but in reality it does not.
> This might be bypassed, but in an expansive and ineffective way. 
> Tested with 2.2.0, 2.1.1 and 2.1.2 spark versions.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org