You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by GitBox <gi...@apache.org> on 2022/01/25 07:53:22 UTC
[GitHub] [avro] ruleeeer opened a new pull request #1473: AVRO-3322:Buffer is not defined in browser environment
ruleeeer opened a new pull request #1473:
URL: https://github.com/apache/avro/pull/1473
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Avro Jira](https://issues.apache.org/jira/browse/AVRO/) issues and references them in the PR title. For example, "AVRO-3322:Buffer is not defined in browser environment"
- https://issues.apache.org/jira/browse/AVRO-3322
- In case you are adding a dependency, check if the license complies with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for this extremely good reason:
- PR simply adds declarations that do not change the actual code logic
### Commits
- [x] My commits all reference Jira issues in their subject lines. In addition, my commits follow the guidelines from "[How to write a good git commit message](https://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes how to use it.
- All the public functions and the classes in the PR contain Javadoc that explain what it does
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [avro] ruleeeer commented on pull request #1473: AVRO-3322: Buffer is not defined in browser environment
Posted by GitBox <gi...@apache.org>.
ruleeeer commented on pull request #1473:
URL: https://github.com/apache/avro/pull/1473#issuecomment-1022818568
@martin-g
When upgrading from webpack4 to webpack5, the `Buffer is not defined` issue will definitely occur when using Avro-browser, because webpack4 has `browserify` built in, which will automatically convert it for you, but webpack5 has to declare `browserify` manually, like Configure it like this
``` javascript
resolve: {
fallback: {
'buffer':require.resolve('buffer')
}
}
```
But that's not enough, you'll still run into the `Buffer is not defined` problem and you'll need to add a little configuration after searching stackoverflow or searching the webpack documentation
``` javascript
plugins: [
new webpack.ProvidePlugin({
Buffer: ['buffer', 'Buffer'],
}),
],
```
This is where you'll find Avro working properly，But this is only `webapck`, there are also projects that use `rollup` or `vite` etc. to complete the application bundle, which will cause everyone using avro-browser to have to search for how to configure to solve the `Buffer is not defined` problem, also, in the official Nodejs documentation it is pointed out that although Buffer hangs globally, but a display declaration is a better approach.You can find it here https://nodejs.org/api/buffer.html#buffer,
<img width="1517" alt="image" src="https://user-images.githubusercontent.com/70385062/151286555-12d37205-92c4-4c79-95d3-289151911476.png">
It only takes two lines to fix the above problem, doesn't it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [avro] ruleeeer commented on pull request #1473: AVRO-3322: Buffer is not defined in browser environment
Posted by GitBox <gi...@apache.org>.
ruleeeer commented on pull request #1473:
URL: https://github.com/apache/avro/pull/1473#issuecomment-1020914651
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [avro] martin-g commented on pull request #1473: AVRO-3322: Buffer is not defined in browser environment
Posted by GitBox <gi...@apache.org>.
martin-g commented on pull request #1473:
URL: https://github.com/apache/avro/pull/1473#issuecomment-1021009227
The simpler, the better!
Bundles would make it harder to understand the errors.
I'd try with Puppeteer.
On Tue, Jan 25, 2022, 10:30 ruleeeer ***@***.***> wrote:
> @martin-g <https://github.com/martin-g> Can I use an E2E testing
> framework to complete this part of the test?, just like
> https://github.com/originjs/vite-plugin-federation/tree/main/packages/examples
> because I need
>
> 1. Browser running environment
> 2. bundle tools(such as rollup or webpack)
> However, this may lead to a larger test project
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/avro/pull/1473#issuecomment-1020927484>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AABYUQRQIN45CZP7AKA6ZILUXZNR5ANCNFSM5MXP7F3Q>
> .
> Triage notifications on the go with GitHub Mobile for iOS
> <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
> or Android
> <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
>
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [avro] martin-g commented on pull request #1473: AVRO-3322: Buffer is not defined in browser environment
Posted by GitBox <gi...@apache.org>.
martin-g commented on pull request #1473:
URL: https://github.com/apache/avro/pull/1473#issuecomment-1020912210
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [avro] martin-g commented on pull request #1473: AVRO-3322: Buffer is not defined in browser environment
Posted by GitBox <gi...@apache.org>.
martin-g commented on pull request #1473:
URL: https://github.com/apache/avro/pull/1473#issuecomment-1020912210
@ruleeeer Thank you for your contribution!
Can you think of some kind of unit test the browser related part of the library to prevent regressions in the future ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [avro] ruleeeer edited a comment on pull request #1473: AVRO-3322: Buffer is not defined in browser environment
Posted by GitBox <gi...@apache.org>.
ruleeeer edited a comment on pull request #1473:
URL: https://github.com/apache/avro/pull/1473#issuecomment-1020927484
@martin-g
Can I use an E2E testing framework and example projects to complete this part of the test?, just like https://github.com/originjs/vite-plugin-federation/tree/main/packages/examples
because I need
1. Browser running environment
2. bundle tools(such as rollup or webpack)
However, this may lead to a larger test project
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [avro] ruleeeer commented on pull request #1473: AVRO-3322: Buffer is not defined in browser environment
Posted by GitBox <gi...@apache.org>.
ruleeeer commented on pull request #1473:
URL: https://github.com/apache/avro/pull/1473#issuecomment-1022969607
Although I think you probably already know this, I'm still making a record here to see how ESM is supported by browsers and nodejs,From what I've looked up, it probably has more of an impact on Nodejs, as I've seen very few issues with browsers using Avro
## Nodejs
<img width="1514" alt="image" src="https://user-images.githubusercontent.com/70385062/151321309-145df66e-9f1a-4c7e-90bc-71c457085e74.png">
## Browser
<img width="1048" alt="image" src="https://user-images.githubusercontent.com/70385062/151321346-098850a9-8db2-4502-bb33-70eb988d65d3.png">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [avro] ruleeeer commented on pull request #1473: AVRO-3322: Buffer is not defined in browser environment
Posted by GitBox <gi...@apache.org>.
ruleeeer commented on pull request #1473:
URL: https://github.com/apache/avro/pull/1473#issuecomment-1022800836
@martin-g
Could you help me? I have a problem with test case.
When I use test frameworks (such as Mocha headless chrome or mochify) to build tests, they all use 'browserify' to bundle the test code (because Avro uses require), but if 'browserify' finds that `require ('buffer ')` is used during bundling, it will automatically mount the `Buffer` to the global attribute, as shown in the figure below. You can find this part of the document here https://github.com/browserify/browserify#compatibility
<img width="856" alt="image" src="https://user-images.githubusercontent.com/70385062/151283397-a017e84f-4918-437c-b442-3f0270d44cce.png">
This means that if we use the "browserify" test library to bundle code, we will never find the problem of "Buffer is not defined". However, Avro browser uses require, we must rely on bundling to convert it into a syntax recognized by the browser. In this way, how should we build test cases? (I can't imagine how to do this without webpack or rollup)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [avro] martin-g commented on pull request #1473: AVRO-3322: Buffer is not defined in browser environment
Posted by GitBox <gi...@apache.org>.
martin-g commented on pull request #1473:
URL: https://github.com/apache/avro/pull/1473#issuecomment-1025451954
I've created https://issues.apache.org/jira/browse/AVRO-3338 for ESM.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [avro] ruleeeer commented on pull request #1473: AVRO-3322: Buffer is not defined in browser environment
Posted by GitBox <gi...@apache.org>.
ruleeeer commented on pull request #1473:
URL: https://github.com/apache/avro/pull/1473#issuecomment-1021105573
I see that the project uses mocha, let me try to use mocha to do this so that I can minimize the added dependencies, or migrating to Puppeteer as you said would be a good option
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [avro] martin-g merged pull request #1473: AVRO-3322: Buffer is not defined in browser environment
Posted by GitBox <gi...@apache.org>.
martin-g merged pull request #1473:
URL: https://github.com/apache/avro/pull/1473
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [avro] ruleeeer edited a comment on pull request #1473: AVRO-3322: Buffer is not defined in browser environment
Posted by GitBox <gi...@apache.org>.
ruleeeer edited a comment on pull request #1473:
URL: https://github.com/apache/avro/pull/1473#issuecomment-1020927484
@martin-g
Can I use an E2E testing framework and example projects to complete this part of the test?, just like https://github.com/originjs/vite-plugin-federation/tree/main/packages/examples
because I need
1. Browser running environment
2. bundle tools(such as rollup or webpack)
However, this may lead to a larger test project
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [avro] martin-g commented on pull request #1473: AVRO-3322: Buffer is not defined in browser environment
Posted by GitBox <gi...@apache.org>.
martin-g commented on pull request #1473:
URL: https://github.com/apache/avro/pull/1473#issuecomment-1022954166
I see!
I think the best would be to use ESM (EcmaScript modules) but I'm not sure
how hard it would be for the existing users (both Node.js and browser ones)
to migrate.
Please use your preferred bundler for the test. I'll investigate the
migration to ESM soon!
On Thu, Jan 27, 2022, 05:42 ruleeeer ***@***.***> wrote:
> @martin-g <https://github.com/martin-g>
> When upgrading from webpack4 to webpack5, the Buffer is not defined issue
> will definitely occur when using Avro-browser, because webpack4 has
> browserify built in, which will automatically convert it for you, but
> webpack5 has to declare browserify manually, like Configure it like this
>
> resolve: {
>
> fallback: {
>
> 'buffer':require.resolve('buffer')
>
> }
>
> }
>
> But that's not enough, you'll still run into the Buffer is not defined
> problem and you'll need to add a little configuration after searching
> stackoverflow or searching the webpack documentation
>
> plugins: [
>
> new webpack.ProvidePlugin({
>
> Buffer: ['buffer', 'Buffer'],
>
> }),
>
> ],
>
> This is where you'll find Avro working properly，But this is only webapck,
> there are also projects that use rollup or vite etc. to complete the
> application bundle, which will cause everyone using avro-browser to have to
> search for how to configure to solve the Buffer is not defined problem,
> also, in the official Nodejs documentation it is pointed out that although
> Buffer hangs globally, but a display declaration is a better approach.You
> can find it here https://nodejs.org/api/buffer.html#buffer,
>
> [image: image]
> <https://user-images.githubusercontent.com/70385062/151286555-12d37205-92c4-4c79-95d3-289151911476.png>
>
> It only takes two lines to fix the above problem, doesn't it?
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/avro/pull/1473#issuecomment-1022818568>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AABYUQS4OXROJ7Q67HZTGWDUYC5JDANCNFSM5MXP7F3Q>
> .
> Triage notifications on the go with GitHub Mobile for iOS
> <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
> or Android
> <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
>
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [avro] martin-g commented on pull request #1473: AVRO-3322: Buffer is not defined in browser environment
Posted by GitBox <gi...@apache.org>.
martin-g commented on pull request #1473:
URL: https://github.com/apache/avro/pull/1473#issuecomment-1023136215
If there is a chance to move to ESM then indeed there is no point in adding
more code now.
On Thu, Jan 27, 2022, 10:43 ruleeeer ***@***.***> wrote:
> Also, is my testing still necessary? (At least so far I haven't found any
> problems with it except for Buffer is not defined) because ESM's testing
> method may be very different from browserify
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/avro/pull/1473#issuecomment-1022974258>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AABYUQUN3NGKV3Z2KZQJBZTUYEATPANCNFSM5MXP7F3Q>
> .
> Triage notifications on the go with GitHub Mobile for iOS
> <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
> or Android
> <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
>
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [avro] martin-g commented on pull request #1473: AVRO-3322: Buffer is not defined in browser environment
Posted by GitBox <gi...@apache.org>.
martin-g commented on pull request #1473:
URL: https://github.com/apache/avro/pull/1473#issuecomment-1025449002
Thank you for the contribution, @ruleeeer !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [avro] ruleeeer commented on pull request #1473: AVRO-3322: Buffer is not defined in browser environment
Posted by GitBox <gi...@apache.org>.
ruleeeer commented on pull request #1473:
URL: https://github.com/apache/avro/pull/1473#issuecomment-1020914651
No problem, let me provide the unit tests for the browser part, but it may take a little time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [avro] ruleeeer commented on pull request #1473: AVRO-3322: Buffer is not defined in browser environment
Posted by GitBox <gi...@apache.org>.
ruleeeer commented on pull request #1473:
URL: https://github.com/apache/avro/pull/1473#issuecomment-1022974258
Also, is my testing still necessary? (At least so far I haven't found any problems with it except for `Buffer is not defined`) because ESM's testing method may be very different from browserify
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [avro] ruleeeer commented on pull request #1473: AVRO-3322: Buffer is not defined in browser environment
Posted by GitBox <gi...@apache.org>.
ruleeeer commented on pull request #1473:
URL: https://github.com/apache/avro/pull/1473#issuecomment-1020927484
@martin-g Can I use an E2E testing framework to complete this part of the test?, just like https://github.com/originjs/vite-plugin-federation/tree/main/packages/examples
because I need
1. Browser running environment
2. bundle tools(such as rollup or webpack)
However, this may lead to a larger test project
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org