You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Robert Weber <mi...@gmail.com> on 2024/09/06 11:41:22 UTC
beam-starter-typescript is broken | long module issue | ttypescript module issue
Hi everyone,
I am very interested in apache beam and watched a video on youtube about
the typescript SDK and that you might need some community support. That's
why I am reaching out. I have a proven experience of 10 years in data
engineering and data analytics, as well as python and javascript/typescript
experience. I am interested to support from time to time. Is this still
relevant?
First of all I tested the starter project for typescript. Unfortunately, it
is broken. I'd like to fix it (
https://beam.apache.org/get-started/quickstart/typescript/). So far I have
some *findings* regarding the *beam-starter-typescript*:
1. The long module is breaking the build:
node_modules/google-gax/node_modules/@grpc/grpc-js/node_modules/long/umd/index.d.ts:1:18
- error TS1479: The current file is a CommonJS module whose imports will
produce 'require' calls; however, the referenced file is an ECMAScript
module and cannot be imported with 'require'. Consider writing a dynamic
'import("../index.js")' call instead.
1 import Long from "../index.js";
To fix this you can either set skibLibCheck to true in tsconfig.json,
which is rather a dirty one, or make sure that an already existing fix (
https://github.com/dcodeIO/long.js/pull/1https://github.com/dcodeIO/long.js/pull/124
24
<https://github.com/dcodeIO/long.js/pull/124>) is implemented in all
dependecies. Therefore, we can simply upgrade all dependecies.
<https://github.com/dcodeIO/long.js/pull/124>
2. After upgrading to typescript to a higher version then >5.0.0, e.g.
version 5.5.4 the ttypescript module is breaking the build:
$ npm run build
> apache-beam-starter-project@0.1.0 build
> ttsc
/home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/lib/patchCreateProgram.js:84
tsm.createProgram = createProgram;
^
TypeError: Cannot set property createProgram of #<Object> which has only
a getter
at patchCreateProgram
(/home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/lib/patchCreateProgram.js:84:23)
at loadTypeScript
(/home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/lib/loadTypescript.js:21:56)
at Object.<anonymous>
(/home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/lib/tsc.js:8:46)
at Module._compile (node:internal/modules/cjs/loader:1364:14)
at Module._extensions..js (node:internal/modules/cjs/loader:1422:10)
at Module.load (node:internal/modules/cjs/loader:1203:32)
at Module._load (node:internal/modules/cjs/loader:1019:12)
at Module.require (node:internal/modules/cjs/loader:1231:19)
at require (node:internal/modules/helpers:177:18)
at Object.<anonymous>
(/home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/bin/tsc:2:1)
There is already a fix available. Thus, when we would move to
ttypescript version 1.1.15 it should be fixed.
https://github.com/nonara/ts-patch/issues/93#issuecomment-1500795063
<https://github.com/nonara/ts-patch/issues/93#issuecomment-1500795063>
I just checked the version on github. In version 2.60-Snapshot the
ttypescript module is already on version 1.1.15. Could you just release a
new npm package. Currently, the newest version on npm is 2.49. Afterwards
we can adapt the apache-beam version in the starter project. I assume then
the starter project would work again.
I am looking forward to hearing from you!
Best,
Robert
Re: beam-starter-typescript is broken | long module issue | ttypescript module issue
Posted by Robert Weber <mi...@gmail.com>.
Oh no, seems like we have one of the legendary domino effects with npm
dependencies... .
I guess we need to tackle them step by step. Currently, I only have an
intel nuc running on linux for private projects. That's why it's a bit
unhandy when you travel by train.
For testings purposes we can use npm link. Basically, it's a two step
process:
1. Go to apache-beam into the typescript folder beam/sdks/typescript and
run:
npm link
This command creates a global npm link on your device. We can now use
the locally developed code directly in a test project.
2. Go to your test project where you want to use the apache-beam package
and run:
npm link apache-beam --save
This should overwrite/add the apache-beam dependency in your
package.json and uses instead the link to your local apache-beam code.
Unfortunately, npm link doesn't behave like a real dependency. That's why
you need to install linked dependencies as well:
npm install --install-links
As I said, I'll test it on the weekend and will give some feedback.
Am Do., 12. Sept. 2024 um 02:57 Uhr schrieb Robert Bradshaw via dev <
dev@beam.apache.org>:
> On Mon, Sep 9, 2024 at 12:00 AM Robert Weber <
> mister.robert.weber@gmail.com> wrote:
>
>> Good morning Roberts!
>>
>> This is such a conversation full of bright fame :-D --> Robert is an old
>> German name that means “bright fame*.” *Just needed to celebrate
>> somehow this coincidence.
>>
>
> :)
>
>
>> This week I am on a business travel.
>>
>
> I've been traveling as well.
>
>
>> But I can prepare something next week and document it as well.
>>
>> I'll send you an update.
>>
>
> So, I took a stab at this. I tried to move to typescript 5, ts-patch, and
> update a bunch of packages, see https://github.com/apache/beam/pull/32439
> . It compiles but is giving import errors (e.g. "Cannot use import
> statement outside a module") I can read and write typescript itself, but
> when it comes to the plethora of modules and packaging structure I am out
> of my depth...
>
>
> https://github.com/apache/beam/pull/32439/commits/b2560be4519a88873e36ce77b084063396c3f896
> was the only interesting (beam-specific) bit. I also had to do
> https://github.com/apache/beam/pull/32439/commits/bfb180d7fdc0ee25d8cd427b77c5ab40deff659c
> Hopefully that should be a good starting point.
>
> (My dev environment is just cloning the repo and running "npm i" in the
> sdks/typescript directory. To test, e.g. the quickstart I run "npm pack"
> and then in the quickstart directory "npm i
> /path/to/created/apache-beam-2.60.0-SNAPSHOT.tgz" Just getting "npm test"
> to work from within sdks/typescript inside the beam repo would be a big
> step forward though.)
>
>
>> Am Sa., 7. Sept. 2024 um 18:55 Uhr schrieb Robert Burke <
>> robert@frantil.com>:
>>
>>> Hello Robert, Robert.
>>>
>>> Since we don't have a great deal of experience with Typescript any
>>> assistance in documenting (or pointing to standard typescript
>>> documentation) on how to hook up a development module to a separate project
>>> that uses the module would be valuable.
>>>
>>> Aside, I should also do the same for the Go side, where the standard is
>>> a "go.work" file that allows the development repo to overlay the
>>> modules usage in a different project.
>>>
>>> Robert
>>>
>>> On Sat, Sep 7, 2024, 12:34 AM Robert Weber <
>>> mister.robert.weber@gmail.com> wrote:
>>>
>>>> I investigated a bit more on the typescript patch compilation topic. It
>>>> seems that the ttypescript module is not well maintained anymore. The last
>>>> version is more than 2 years old. An alternative is ts-patch. This one
>>>> seems to be maintained and working with typescript version >5.0.0. Hence, I
>>>> suggest a migration to ts-patch.
>>>> ts-patch migration steps:
>>>>
>>>> 1. install ts-patch
>>>> 2. use tspc instead of ttsc to compile during build (build.sh)
>>>> [image: image.png]
>>>> 3. uninstall ttypescript
>>>>
>>>> I think we don't need to adapt the transformer plugins in
>>>> tsconfig.json, since, the plugin options are the same as for ttypescript.
>>>> Hope this works!
>>>>
>>>> I definitely need to setup my own dev environment. So I can build and
>>>> link apache-beam to the starter project locally and test it on my own. I
>>>> will do this as my next step :-).
>>>>
>>>> Am Fr., 6. Sept. 2024 um 18:24 Uhr schrieb Robert Bradshaw via dev <
>>>> dev@beam.apache.org>:
>>>>
>>>>> On Fri, Sep 6, 2024 at 4:41 AM Robert Weber
>>>>> <mi...@gmail.com> wrote:
>>>>> >
>>>>> > Hi everyone,
>>>>> >
>>>>> > I am very interested in apache beam and watched a video on youtube
>>>>> about the typescript SDK and that you might need some community support.
>>>>>
>>>>> Yes, definitely.
>>>>>
>>>>> > That's why I am reaching out.
>>>>>
>>>>> That'd be great!
>>>>>
>>>>> > I have a proven experience of 10 years in data engineering and data
>>>>> analytics, as well as python and javascript/typescript experience. I am
>>>>> interested to support from time to time. Is this still relevant?
>>>>> >
>>>>> > First of all I tested the starter project for typescript.
>>>>> Unfortunately, it is broken. I'd like to fix it (
>>>>> https://beam.apache.org/get-started/quickstart/typescript/). So far I
>>>>> have some findings regarding the beam-starter-typescript:
>>>>> >
>>>>> > The long module is breaking the build:
>>>>> >
>>>>> >
>>>>> node_modules/google-gax/node_modules/@grpc/grpc-js/node_modules/long/umd/index.d.ts:1:18
>>>>> - error TS1479: The current file is a CommonJS module whose imports will
>>>>> produce 'require' calls; however, the referenced file is an ECMAScript
>>>>> module and cannot be imported with 'require'. Consider writing a dynamic
>>>>> 'import("../index.js")' call instead.
>>>>> >
>>>>> > 1 import Long from "../index.js";
>>>>> >
>>>>> > To fix this you can either set skibLibCheck to true in
>>>>> tsconfig.json, which is rather a dirty one, or make sure that an already
>>>>> existing fix (
>>>>> https://github.com/dcodeIO/long.js/pull/1https://github.com/dcodeIO/long.js/pull/124
>>>>> 24) is implemented in all dependecies. Therefore, we can simply upgrade all
>>>>> dependecies.
>>>>> >
>>>>> > After upgrading to typescript to a higher version then >5.0.0, e.g.
>>>>> version 5.5.4 the ttypescript module is breaking the build:
>>>>> >
>>>>> > $ npm run build
>>>>> >
>>>>> > > apache-beam-starter-project@0.1.0 build
>>>>> > > ttsc
>>>>> >
>>>>> >
>>>>> /home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/lib/patchCreateProgram.js:84
>>>>> > tsm.createProgram = createProgram;
>>>>> > ^
>>>>> >
>>>>> > TypeError: Cannot set property createProgram of #<Object> which has
>>>>> only a getter
>>>>> > at patchCreateProgram
>>>>> (/home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/lib/patchCreateProgram.js:84:23)
>>>>> > at loadTypeScript
>>>>> (/home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/lib/loadTypescript.js:21:56)
>>>>> > at Object.<anonymous>
>>>>> (/home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/lib/tsc.js:8:46)
>>>>> > at Module._compile (node:internal/modules/cjs/loader:1364:14)
>>>>> > at Module._extensions..js
>>>>> (node:internal/modules/cjs/loader:1422:10)
>>>>> > at Module.load (node:internal/modules/cjs/loader:1203:32)
>>>>> > at Module._load (node:internal/modules/cjs/loader:1019:12)
>>>>> > at Module.require (node:internal/modules/cjs/loader:1231:19)
>>>>> > at require (node:internal/modules/helpers:177:18)
>>>>> > at Object.<anonymous>
>>>>> (/home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/bin/tsc:2:1)
>>>>> >
>>>>> > There is already a fix available. Thus, when we would move to
>>>>> ttypescript version 1.1.15 it should be fixed.
>>>>> > https://github.com/nonara/ts-patch/issues/93#issuecomment-1500795063
>>>>> >
>>>>> > I just checked the version on github. In version 2.60-Snapshot the
>>>>> ttypescript module is already on version 1.1.15. Could you just release a
>>>>> new npm package.
>>>>>
>>>>> I tried upgrading typescript to 5.5.4,
>>>>>
>>>>> https://github.com/apache/beam/pull/32405/files
>>>>>
>>>>> Despite ttypescript being 1.1.15, I'm stiil getting the error above
>>>>> when trying to build.
>>>>>
>>>>> I'll admit this is the area that I am the least comfortable with,
>>>>> difficulties with packaging and versioning (and the various kinds of
>>>>> modules) is what prevented me from releasing new mpms in the first
>>>>> place (and then I got busy with other stuff and never got back to
>>>>> this). Would greatly appreciate any help here!
>>>>>
>>>>> > Currently, the newest version on npm is 2.49. Afterwards we can
>>>>> adapt the apache-beam version in the starter project. I assume then the
>>>>> starter project would work again.
>>>>> >
>>>>> > I am looking forward to hearing from you!
>>>>> >
>>>>> > Best,
>>>>> > Robert
>>>>> >
>>>>>
>>>>
Re: beam-starter-typescript is broken | long module issue | ttypescript module issue
Posted by Robert Bradshaw via dev <de...@beam.apache.org>.
On Mon, Sep 9, 2024 at 12:00 AM Robert Weber <mi...@gmail.com>
wrote:
> Good morning Roberts!
>
> This is such a conversation full of bright fame :-D --> Robert is an old
> German name that means “bright fame*.” *Just needed to celebrate somehow
> this coincidence.
>
:)
> This week I am on a business travel.
>
I've been traveling as well.
> But I can prepare something next week and document it as well.
>
> I'll send you an update.
>
So, I took a stab at this. I tried to move to typescript 5, ts-patch, and
update a bunch of packages, see https://github.com/apache/beam/pull/32439 .
It compiles but is giving import errors (e.g. "Cannot use import statement
outside a module") I can read and write typescript itself, but when it
comes to the plethora of modules and packaging structure I am out of my
depth...
https://github.com/apache/beam/pull/32439/commits/b2560be4519a88873e36ce77b084063396c3f896
was the only interesting (beam-specific) bit. I also had to do
https://github.com/apache/beam/pull/32439/commits/bfb180d7fdc0ee25d8cd427b77c5ab40deff659c
Hopefully that should be a good starting point.
(My dev environment is just cloning the repo and running "npm i" in the
sdks/typescript directory. To test, e.g. the quickstart I run "npm pack"
and then in the quickstart directory "npm i
/path/to/created/apache-beam-2.60.0-SNAPSHOT.tgz" Just getting "npm test"
to work from within sdks/typescript inside the beam repo would be a big
step forward though.)
> Am Sa., 7. Sept. 2024 um 18:55 Uhr schrieb Robert Burke <
> robert@frantil.com>:
>
>> Hello Robert, Robert.
>>
>> Since we don't have a great deal of experience with Typescript any
>> assistance in documenting (or pointing to standard typescript
>> documentation) on how to hook up a development module to a separate project
>> that uses the module would be valuable.
>>
>> Aside, I should also do the same for the Go side, where the standard is a
>> "go.work" file that allows the development repo to overlay the modules
>> usage in a different project.
>>
>> Robert
>>
>> On Sat, Sep 7, 2024, 12:34 AM Robert Weber <mi...@gmail.com>
>> wrote:
>>
>>> I investigated a bit more on the typescript patch compilation topic. It
>>> seems that the ttypescript module is not well maintained anymore. The last
>>> version is more than 2 years old. An alternative is ts-patch. This one
>>> seems to be maintained and working with typescript version >5.0.0. Hence, I
>>> suggest a migration to ts-patch.
>>> ts-patch migration steps:
>>>
>>> 1. install ts-patch
>>> 2. use tspc instead of ttsc to compile during build (build.sh)
>>> [image: image.png]
>>> 3. uninstall ttypescript
>>>
>>> I think we don't need to adapt the transformer plugins in tsconfig.json,
>>> since, the plugin options are the same as for ttypescript. Hope this works!
>>>
>>> I definitely need to setup my own dev environment. So I can build and
>>> link apache-beam to the starter project locally and test it on my own. I
>>> will do this as my next step :-).
>>>
>>> Am Fr., 6. Sept. 2024 um 18:24 Uhr schrieb Robert Bradshaw via dev <
>>> dev@beam.apache.org>:
>>>
>>>> On Fri, Sep 6, 2024 at 4:41 AM Robert Weber
>>>> <mi...@gmail.com> wrote:
>>>> >
>>>> > Hi everyone,
>>>> >
>>>> > I am very interested in apache beam and watched a video on youtube
>>>> about the typescript SDK and that you might need some community support.
>>>>
>>>> Yes, definitely.
>>>>
>>>> > That's why I am reaching out.
>>>>
>>>> That'd be great!
>>>>
>>>> > I have a proven experience of 10 years in data engineering and data
>>>> analytics, as well as python and javascript/typescript experience. I am
>>>> interested to support from time to time. Is this still relevant?
>>>> >
>>>> > First of all I tested the starter project for typescript.
>>>> Unfortunately, it is broken. I'd like to fix it (
>>>> https://beam.apache.org/get-started/quickstart/typescript/). So far I
>>>> have some findings regarding the beam-starter-typescript:
>>>> >
>>>> > The long module is breaking the build:
>>>> >
>>>> >
>>>> node_modules/google-gax/node_modules/@grpc/grpc-js/node_modules/long/umd/index.d.ts:1:18
>>>> - error TS1479: The current file is a CommonJS module whose imports will
>>>> produce 'require' calls; however, the referenced file is an ECMAScript
>>>> module and cannot be imported with 'require'. Consider writing a dynamic
>>>> 'import("../index.js")' call instead.
>>>> >
>>>> > 1 import Long from "../index.js";
>>>> >
>>>> > To fix this you can either set skibLibCheck to true in tsconfig.json,
>>>> which is rather a dirty one, or make sure that an already existing fix (
>>>> https://github.com/dcodeIO/long.js/pull/1https://github.com/dcodeIO/long.js/pull/124
>>>> 24) is implemented in all dependecies. Therefore, we can simply upgrade all
>>>> dependecies.
>>>> >
>>>> > After upgrading to typescript to a higher version then >5.0.0, e.g.
>>>> version 5.5.4 the ttypescript module is breaking the build:
>>>> >
>>>> > $ npm run build
>>>> >
>>>> > > apache-beam-starter-project@0.1.0 build
>>>> > > ttsc
>>>> >
>>>> >
>>>> /home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/lib/patchCreateProgram.js:84
>>>> > tsm.createProgram = createProgram;
>>>> > ^
>>>> >
>>>> > TypeError: Cannot set property createProgram of #<Object> which has
>>>> only a getter
>>>> > at patchCreateProgram
>>>> (/home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/lib/patchCreateProgram.js:84:23)
>>>> > at loadTypeScript
>>>> (/home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/lib/loadTypescript.js:21:56)
>>>> > at Object.<anonymous>
>>>> (/home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/lib/tsc.js:8:46)
>>>> > at Module._compile (node:internal/modules/cjs/loader:1364:14)
>>>> > at Module._extensions..js
>>>> (node:internal/modules/cjs/loader:1422:10)
>>>> > at Module.load (node:internal/modules/cjs/loader:1203:32)
>>>> > at Module._load (node:internal/modules/cjs/loader:1019:12)
>>>> > at Module.require (node:internal/modules/cjs/loader:1231:19)
>>>> > at require (node:internal/modules/helpers:177:18)
>>>> > at Object.<anonymous>
>>>> (/home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/bin/tsc:2:1)
>>>> >
>>>> > There is already a fix available. Thus, when we would move to
>>>> ttypescript version 1.1.15 it should be fixed.
>>>> > https://github.com/nonara/ts-patch/issues/93#issuecomment-1500795063
>>>> >
>>>> > I just checked the version on github. In version 2.60-Snapshot the
>>>> ttypescript module is already on version 1.1.15. Could you just release a
>>>> new npm package.
>>>>
>>>> I tried upgrading typescript to 5.5.4,
>>>>
>>>> https://github.com/apache/beam/pull/32405/files
>>>>
>>>> Despite ttypescript being 1.1.15, I'm stiil getting the error above
>>>> when trying to build.
>>>>
>>>> I'll admit this is the area that I am the least comfortable with,
>>>> difficulties with packaging and versioning (and the various kinds of
>>>> modules) is what prevented me from releasing new mpms in the first
>>>> place (and then I got busy with other stuff and never got back to
>>>> this). Would greatly appreciate any help here!
>>>>
>>>> > Currently, the newest version on npm is 2.49. Afterwards we can adapt
>>>> the apache-beam version in the starter project. I assume then the starter
>>>> project would work again.
>>>> >
>>>> > I am looking forward to hearing from you!
>>>> >
>>>> > Best,
>>>> > Robert
>>>> >
>>>>
>>>
Re: beam-starter-typescript is broken | long module issue | ttypescript module issue
Posted by Robert Weber <mi...@gmail.com>.
Good morning Roberts!
This is such a conversation full of bright fame :-D --> Robert is an old
German name that means “bright fame*.” *Just needed to celebrate somehow
this coincidence.
This week I am on a business travel. But I can prepare something next week
and document it as well.
I'll send you an update.
Best,
Robert
Am Sa., 7. Sept. 2024 um 18:55 Uhr schrieb Robert Burke <robert@frantil.com
>:
> Hello Robert, Robert.
>
> Since we don't have a great deal of experience with Typescript any
> assistance in documenting (or pointing to standard typescript
> documentation) on how to hook up a development module to a separate project
> that uses the module would be valuable.
>
> Aside, I should also do the same for the Go side, where the standard is a "
> go.work" file that allows the development repo to overlay the modules
> usage in a different project.
>
> Robert
>
> On Sat, Sep 7, 2024, 12:34 AM Robert Weber <mi...@gmail.com>
> wrote:
>
>> I investigated a bit more on the typescript patch compilation topic. It
>> seems that the ttypescript module is not well maintained anymore. The last
>> version is more than 2 years old. An alternative is ts-patch. This one
>> seems to be maintained and working with typescript version >5.0.0. Hence, I
>> suggest a migration to ts-patch.
>> ts-patch migration steps:
>>
>> 1. install ts-patch
>> 2. use tspc instead of ttsc to compile during build (build.sh)
>> [image: image.png]
>> 3. uninstall ttypescript
>>
>> I think we don't need to adapt the transformer plugins in tsconfig.json,
>> since, the plugin options are the same as for ttypescript. Hope this works!
>>
>> I definitely need to setup my own dev environment. So I can build and
>> link apache-beam to the starter project locally and test it on my own. I
>> will do this as my next step :-).
>>
>> Am Fr., 6. Sept. 2024 um 18:24 Uhr schrieb Robert Bradshaw via dev <
>> dev@beam.apache.org>:
>>
>>> On Fri, Sep 6, 2024 at 4:41 AM Robert Weber
>>> <mi...@gmail.com> wrote:
>>> >
>>> > Hi everyone,
>>> >
>>> > I am very interested in apache beam and watched a video on youtube
>>> about the typescript SDK and that you might need some community support.
>>>
>>> Yes, definitely.
>>>
>>> > That's why I am reaching out.
>>>
>>> That'd be great!
>>>
>>> > I have a proven experience of 10 years in data engineering and data
>>> analytics, as well as python and javascript/typescript experience. I am
>>> interested to support from time to time. Is this still relevant?
>>> >
>>> > First of all I tested the starter project for typescript.
>>> Unfortunately, it is broken. I'd like to fix it (
>>> https://beam.apache.org/get-started/quickstart/typescript/). So far I
>>> have some findings regarding the beam-starter-typescript:
>>> >
>>> > The long module is breaking the build:
>>> >
>>> >
>>> node_modules/google-gax/node_modules/@grpc/grpc-js/node_modules/long/umd/index.d.ts:1:18
>>> - error TS1479: The current file is a CommonJS module whose imports will
>>> produce 'require' calls; however, the referenced file is an ECMAScript
>>> module and cannot be imported with 'require'. Consider writing a dynamic
>>> 'import("../index.js")' call instead.
>>> >
>>> > 1 import Long from "../index.js";
>>> >
>>> > To fix this you can either set skibLibCheck to true in tsconfig.json,
>>> which is rather a dirty one, or make sure that an already existing fix (
>>> https://github.com/dcodeIO/long.js/pull/1https://github.com/dcodeIO/long.js/pull/124
>>> 24) is implemented in all dependecies. Therefore, we can simply upgrade all
>>> dependecies.
>>> >
>>> > After upgrading to typescript to a higher version then >5.0.0, e.g.
>>> version 5.5.4 the ttypescript module is breaking the build:
>>> >
>>> > $ npm run build
>>> >
>>> > > apache-beam-starter-project@0.1.0 build
>>> > > ttsc
>>> >
>>> >
>>> /home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/lib/patchCreateProgram.js:84
>>> > tsm.createProgram = createProgram;
>>> > ^
>>> >
>>> > TypeError: Cannot set property createProgram of #<Object> which has
>>> only a getter
>>> > at patchCreateProgram
>>> (/home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/lib/patchCreateProgram.js:84:23)
>>> > at loadTypeScript
>>> (/home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/lib/loadTypescript.js:21:56)
>>> > at Object.<anonymous>
>>> (/home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/lib/tsc.js:8:46)
>>> > at Module._compile (node:internal/modules/cjs/loader:1364:14)
>>> > at Module._extensions..js
>>> (node:internal/modules/cjs/loader:1422:10)
>>> > at Module.load (node:internal/modules/cjs/loader:1203:32)
>>> > at Module._load (node:internal/modules/cjs/loader:1019:12)
>>> > at Module.require (node:internal/modules/cjs/loader:1231:19)
>>> > at require (node:internal/modules/helpers:177:18)
>>> > at Object.<anonymous>
>>> (/home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/bin/tsc:2:1)
>>> >
>>> > There is already a fix available. Thus, when we would move to
>>> ttypescript version 1.1.15 it should be fixed.
>>> > https://github.com/nonara/ts-patch/issues/93#issuecomment-1500795063
>>> >
>>> > I just checked the version on github. In version 2.60-Snapshot the
>>> ttypescript module is already on version 1.1.15. Could you just release a
>>> new npm package.
>>>
>>> I tried upgrading typescript to 5.5.4,
>>>
>>> https://github.com/apache/beam/pull/32405/files
>>>
>>> Despite ttypescript being 1.1.15, I'm stiil getting the error above
>>> when trying to build.
>>>
>>> I'll admit this is the area that I am the least comfortable with,
>>> difficulties with packaging and versioning (and the various kinds of
>>> modules) is what prevented me from releasing new mpms in the first
>>> place (and then I got busy with other stuff and never got back to
>>> this). Would greatly appreciate any help here!
>>>
>>> > Currently, the newest version on npm is 2.49. Afterwards we can adapt
>>> the apache-beam version in the starter project. I assume then the starter
>>> project would work again.
>>> >
>>> > I am looking forward to hearing from you!
>>> >
>>> > Best,
>>> > Robert
>>> >
>>>
>>
Re: beam-starter-typescript is broken | long module issue | ttypescript module issue
Posted by Robert Burke <ro...@frantil.com>.
Hello Robert, Robert.
Since we don't have a great deal of experience with Typescript any
assistance in documenting (or pointing to standard typescript
documentation) on how to hook up a development module to a separate project
that uses the module would be valuable.
Aside, I should also do the same for the Go side, where the standard is a "
go.work" file that allows the development repo to overlay the modules usage
in a different project.
Robert
On Sat, Sep 7, 2024, 12:34 AM Robert Weber <mi...@gmail.com>
wrote:
> I investigated a bit more on the typescript patch compilation topic. It
> seems that the ttypescript module is not well maintained anymore. The last
> version is more than 2 years old. An alternative is ts-patch. This one
> seems to be maintained and working with typescript version >5.0.0. Hence, I
> suggest a migration to ts-patch.
> ts-patch migration steps:
>
> 1. install ts-patch
> 2. use tspc instead of ttsc to compile during build (build.sh)
> [image: image.png]
> 3. uninstall ttypescript
>
> I think we don't need to adapt the transformer plugins in tsconfig.json,
> since, the plugin options are the same as for ttypescript. Hope this works!
>
> I definitely need to setup my own dev environment. So I can build and link
> apache-beam to the starter project locally and test it on my own. I will do
> this as my next step :-).
>
> Am Fr., 6. Sept. 2024 um 18:24 Uhr schrieb Robert Bradshaw via dev <
> dev@beam.apache.org>:
>
>> On Fri, Sep 6, 2024 at 4:41 AM Robert Weber
>> <mi...@gmail.com> wrote:
>> >
>> > Hi everyone,
>> >
>> > I am very interested in apache beam and watched a video on youtube
>> about the typescript SDK and that you might need some community support.
>>
>> Yes, definitely.
>>
>> > That's why I am reaching out.
>>
>> That'd be great!
>>
>> > I have a proven experience of 10 years in data engineering and data
>> analytics, as well as python and javascript/typescript experience. I am
>> interested to support from time to time. Is this still relevant?
>> >
>> > First of all I tested the starter project for typescript.
>> Unfortunately, it is broken. I'd like to fix it (
>> https://beam.apache.org/get-started/quickstart/typescript/). So far I
>> have some findings regarding the beam-starter-typescript:
>> >
>> > The long module is breaking the build:
>> >
>> >
>> node_modules/google-gax/node_modules/@grpc/grpc-js/node_modules/long/umd/index.d.ts:1:18
>> - error TS1479: The current file is a CommonJS module whose imports will
>> produce 'require' calls; however, the referenced file is an ECMAScript
>> module and cannot be imported with 'require'. Consider writing a dynamic
>> 'import("../index.js")' call instead.
>> >
>> > 1 import Long from "../index.js";
>> >
>> > To fix this you can either set skibLibCheck to true in tsconfig.json,
>> which is rather a dirty one, or make sure that an already existing fix (
>> https://github.com/dcodeIO/long.js/pull/1https://github.com/dcodeIO/long.js/pull/124
>> 24) is implemented in all dependecies. Therefore, we can simply upgrade all
>> dependecies.
>> >
>> > After upgrading to typescript to a higher version then >5.0.0, e.g.
>> version 5.5.4 the ttypescript module is breaking the build:
>> >
>> > $ npm run build
>> >
>> > > apache-beam-starter-project@0.1.0 build
>> > > ttsc
>> >
>> >
>> /home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/lib/patchCreateProgram.js:84
>> > tsm.createProgram = createProgram;
>> > ^
>> >
>> > TypeError: Cannot set property createProgram of #<Object> which has
>> only a getter
>> > at patchCreateProgram
>> (/home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/lib/patchCreateProgram.js:84:23)
>> > at loadTypeScript
>> (/home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/lib/loadTypescript.js:21:56)
>> > at Object.<anonymous>
>> (/home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/lib/tsc.js:8:46)
>> > at Module._compile (node:internal/modules/cjs/loader:1364:14)
>> > at Module._extensions..js (node:internal/modules/cjs/loader:1422:10)
>> > at Module.load (node:internal/modules/cjs/loader:1203:32)
>> > at Module._load (node:internal/modules/cjs/loader:1019:12)
>> > at Module.require (node:internal/modules/cjs/loader:1231:19)
>> > at require (node:internal/modules/helpers:177:18)
>> > at Object.<anonymous>
>> (/home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/bin/tsc:2:1)
>> >
>> > There is already a fix available. Thus, when we would move to
>> ttypescript version 1.1.15 it should be fixed.
>> > https://github.com/nonara/ts-patch/issues/93#issuecomment-1500795063
>> >
>> > I just checked the version on github. In version 2.60-Snapshot the
>> ttypescript module is already on version 1.1.15. Could you just release a
>> new npm package.
>>
>> I tried upgrading typescript to 5.5.4,
>>
>> https://github.com/apache/beam/pull/32405/files
>>
>> Despite ttypescript being 1.1.15, I'm stiil getting the error above
>> when trying to build.
>>
>> I'll admit this is the area that I am the least comfortable with,
>> difficulties with packaging and versioning (and the various kinds of
>> modules) is what prevented me from releasing new mpms in the first
>> place (and then I got busy with other stuff and never got back to
>> this). Would greatly appreciate any help here!
>>
>> > Currently, the newest version on npm is 2.49. Afterwards we can adapt
>> the apache-beam version in the starter project. I assume then the starter
>> project would work again.
>> >
>> > I am looking forward to hearing from you!
>> >
>> > Best,
>> > Robert
>> >
>>
>
Re: beam-starter-typescript is broken | long module issue | ttypescript module issue
Posted by Robert Weber <mi...@gmail.com>.
I investigated a bit more on the typescript patch compilation topic. It
seems that the ttypescript module is not well maintained anymore. The last
version is more than 2 years old. An alternative is ts-patch. This one
seems to be maintained and working with typescript version >5.0.0. Hence, I
suggest a migration to ts-patch.
ts-patch migration steps:
1. install ts-patch
2. use tspc instead of ttsc to compile during build (build.sh)
[image: image.png]
3. uninstall ttypescript
I think we don't need to adapt the transformer plugins in tsconfig.json,
since, the plugin options are the same as for ttypescript. Hope this works!
I definitely need to setup my own dev environment. So I can build and link
apache-beam to the starter project locally and test it on my own. I will do
this as my next step :-).
Am Fr., 6. Sept. 2024 um 18:24 Uhr schrieb Robert Bradshaw via dev <
dev@beam.apache.org>:
> On Fri, Sep 6, 2024 at 4:41 AM Robert Weber
> <mi...@gmail.com> wrote:
> >
> > Hi everyone,
> >
> > I am very interested in apache beam and watched a video on youtube about
> the typescript SDK and that you might need some community support.
>
> Yes, definitely.
>
> > That's why I am reaching out.
>
> That'd be great!
>
> > I have a proven experience of 10 years in data engineering and data
> analytics, as well as python and javascript/typescript experience. I am
> interested to support from time to time. Is this still relevant?
> >
> > First of all I tested the starter project for typescript. Unfortunately,
> it is broken. I'd like to fix it (
> https://beam.apache.org/get-started/quickstart/typescript/). So far I
> have some findings regarding the beam-starter-typescript:
> >
> > The long module is breaking the build:
> >
> >
> node_modules/google-gax/node_modules/@grpc/grpc-js/node_modules/long/umd/index.d.ts:1:18
> - error TS1479: The current file is a CommonJS module whose imports will
> produce 'require' calls; however, the referenced file is an ECMAScript
> module and cannot be imported with 'require'. Consider writing a dynamic
> 'import("../index.js")' call instead.
> >
> > 1 import Long from "../index.js";
> >
> > To fix this you can either set skibLibCheck to true in tsconfig.json,
> which is rather a dirty one, or make sure that an already existing fix (
> https://github.com/dcodeIO/long.js/pull/1https://github.com/dcodeIO/long.js/pull/124
> 24) is implemented in all dependecies. Therefore, we can simply upgrade all
> dependecies.
> >
> > After upgrading to typescript to a higher version then >5.0.0, e.g.
> version 5.5.4 the ttypescript module is breaking the build:
> >
> > $ npm run build
> >
> > > apache-beam-starter-project@0.1.0 build
> > > ttsc
> >
> >
> /home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/lib/patchCreateProgram.js:84
> > tsm.createProgram = createProgram;
> > ^
> >
> > TypeError: Cannot set property createProgram of #<Object> which has only
> a getter
> > at patchCreateProgram
> (/home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/lib/patchCreateProgram.js:84:23)
> > at loadTypeScript
> (/home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/lib/loadTypescript.js:21:56)
> > at Object.<anonymous>
> (/home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/lib/tsc.js:8:46)
> > at Module._compile (node:internal/modules/cjs/loader:1364:14)
> > at Module._extensions..js (node:internal/modules/cjs/loader:1422:10)
> > at Module.load (node:internal/modules/cjs/loader:1203:32)
> > at Module._load (node:internal/modules/cjs/loader:1019:12)
> > at Module.require (node:internal/modules/cjs/loader:1231:19)
> > at require (node:internal/modules/helpers:177:18)
> > at Object.<anonymous>
> (/home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/bin/tsc:2:1)
> >
> > There is already a fix available. Thus, when we would move to
> ttypescript version 1.1.15 it should be fixed.
> > https://github.com/nonara/ts-patch/issues/93#issuecomment-1500795063
> >
> > I just checked the version on github. In version 2.60-Snapshot the
> ttypescript module is already on version 1.1.15. Could you just release a
> new npm package.
>
> I tried upgrading typescript to 5.5.4,
>
> https://github.com/apache/beam/pull/32405/files
>
> Despite ttypescript being 1.1.15, I'm stiil getting the error above
> when trying to build.
>
> I'll admit this is the area that I am the least comfortable with,
> difficulties with packaging and versioning (and the various kinds of
> modules) is what prevented me from releasing new mpms in the first
> place (and then I got busy with other stuff and never got back to
> this). Would greatly appreciate any help here!
>
> > Currently, the newest version on npm is 2.49. Afterwards we can adapt
> the apache-beam version in the starter project. I assume then the starter
> project would work again.
> >
> > I am looking forward to hearing from you!
> >
> > Best,
> > Robert
> >
>
Re: beam-starter-typescript is broken | long module issue | ttypescript module issue
Posted by Robert Bradshaw via dev <de...@beam.apache.org>.
On Fri, Sep 6, 2024 at 4:41 AM Robert Weber
<mi...@gmail.com> wrote:
>
> Hi everyone,
>
> I am very interested in apache beam and watched a video on youtube about the typescript SDK and that you might need some community support.
Yes, definitely.
> That's why I am reaching out.
That'd be great!
> I have a proven experience of 10 years in data engineering and data analytics, as well as python and javascript/typescript experience. I am interested to support from time to time. Is this still relevant?
>
> First of all I tested the starter project for typescript. Unfortunately, it is broken. I'd like to fix it (https://beam.apache.org/get-started/quickstart/typescript/). So far I have some findings regarding the beam-starter-typescript:
>
> The long module is breaking the build:
>
> node_modules/google-gax/node_modules/@grpc/grpc-js/node_modules/long/umd/index.d.ts:1:18 - error TS1479: The current file is a CommonJS module whose imports will produce 'require' calls; however, the referenced file is an ECMAScript module and cannot be imported with 'require'. Consider writing a dynamic 'import("../index.js")' call instead.
>
> 1 import Long from "../index.js";
>
> To fix this you can either set skibLibCheck to true in tsconfig.json, which is rather a dirty one, or make sure that an already existing fix (https://github.com/dcodeIO/long.js/pull/1https://github.com/dcodeIO/long.js/pull/124 24) is implemented in all dependecies. Therefore, we can simply upgrade all dependecies.
>
> After upgrading to typescript to a higher version then >5.0.0, e.g. version 5.5.4 the ttypescript module is breaking the build:
>
> $ npm run build
>
> > apache-beam-starter-project@0.1.0 build
> > ttsc
>
> /home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/lib/patchCreateProgram.js:84
> tsm.createProgram = createProgram;
> ^
>
> TypeError: Cannot set property createProgram of #<Object> which has only a getter
> at patchCreateProgram (/home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/lib/patchCreateProgram.js:84:23)
> at loadTypeScript (/home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/lib/loadTypescript.js:21:56)
> at Object.<anonymous> (/home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/lib/tsc.js:8:46)
> at Module._compile (node:internal/modules/cjs/loader:1364:14)
> at Module._extensions..js (node:internal/modules/cjs/loader:1422:10)
> at Module.load (node:internal/modules/cjs/loader:1203:32)
> at Module._load (node:internal/modules/cjs/loader:1019:12)
> at Module.require (node:internal/modules/cjs/loader:1231:19)
> at require (node:internal/modules/helpers:177:18)
> at Object.<anonymous> (/home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/bin/tsc:2:1)
>
> There is already a fix available. Thus, when we would move to ttypescript version 1.1.15 it should be fixed.
> https://github.com/nonara/ts-patch/issues/93#issuecomment-1500795063
>
> I just checked the version on github. In version 2.60-Snapshot the ttypescript module is already on version 1.1.15. Could you just release a new npm package.
I tried upgrading typescript to 5.5.4,
https://github.com/apache/beam/pull/32405/files
Despite ttypescript being 1.1.15, I'm stiil getting the error above
when trying to build.
I'll admit this is the area that I am the least comfortable with,
difficulties with packaging and versioning (and the various kinds of
modules) is what prevented me from releasing new mpms in the first
place (and then I got busy with other stuff and never got back to
this). Would greatly appreciate any help here!
> Currently, the newest version on npm is 2.49. Afterwards we can adapt the apache-beam version in the starter project. I assume then the starter project would work again.
>
> I am looking forward to hearing from you!
>
> Best,
> Robert
>