You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nuttx.apache.org by Qi3 Huang 黄齐 <hu...@xiaomi.com.INVALID> on 2023/03/10 15:47:43 UTC
答复: 答复: [External Mail]Re: [Breaking change] Echo character by serial driver
Emm, I confirmed that one of my test platform (esp32c3), its cdcacm driver implement the uart_ops, but rp2040 seems not.
CLE should works since it will override the content from driver ECHO if over serial driver, but readline shouldn't echo input.
I'll test more to confirm it.
________________________________
发件人: Gregory Nutt <sp...@gmail.com>
发送时间: 2023年3月10日 23:22:45
收件人: dev@nuttx.apache.org
主题: Re: 答复: [External Mail]Re: [Breaking change] Echo character by serial driver
[外部邮件] 此邮件来源于小米公司外部，请谨慎处理。若对邮件安全性存疑，请将邮件转发给misec@xiaomi.com进行反馈
> nsh over usb works with readline/cle now.
>
>
> In face, isconsole in current implementation is more like a very tiny `line driver` inside the serial driver.
That is very interesting. The serial driver (drivers/usbdev/cdcacm.c).
Does not include any of your change. It does not use serial.c . It has
its own serial interface and its own TERMIOS implementation that does
not use your changes.
So I would expect serial.c to be better behaved without the ECHO as well.
#/******本邮件及其附件含有小米公司的保密信息，仅限于发送给上面地址中列出的个人或群组。禁止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制、或散发）本邮件中的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并删除本邮件！ This e-mail and its attachments contain confidential information from XIAOMI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it!******/#
Re: 答复: 答复: 答复: [External Mail]Re: [Breaking change] Echo character by serial driver
Posted by Gregory Nutt <sp...@gmail.com>.
On 3/10/2023 10:05 AM, Qi3 Huang 黄齐 wrote:
> Thanks for point that, so my change will affect them by serial.c since it's in common logic (excatly, is in uart_read()).
You are right. But that is not true of telnet, is it?
Re: 答复: 答复: 答复: [External Mail]Re: [Breaking change] Echo character by serial driver
Posted by Gregory Nutt <sp...@gmail.com>.
It is true that Linux is not fully POSIX compliant (e.g.,
https://linuxhint.com/is_linux_posix_compliant/). But Linux folk have
gone to a lot of work to clean up a Linux specification with the LSB:
https://refspecs.linuxfoundation.org/lsb.shtml and that generally adopts
POSIX standards.
We use the the term "Linux" loosely here. POSIX specifies the
interface between applications and an operating system. However,
the user application never interacts directly with Linux which
resides in kernel space but rather with various user space libraries
like libc that, in turn, interface with Linux via call gates
(uncontrolled and not not prototype-able C interfaces). So more
correctly, it is the GNU libraries that determine the user
application interface. Calling this Gnu/Linux is more accurate but
that phrase manages to piss off everyone.
NuttX has evolved over the years in regard to what it uses as its
standards base.
1. Originally, in the beginning, NuttX was a very tiny OS (think
FreeRTOS or ChibiOS) with a very limited user interface but drawing
primarily from POSIX interface definitions. Since NuttX provides both
the OS and the libc, we can safely call this interface NuttX.
2. Later the project adopted OpenGroup.org as the standards base.
OpenGroup.org is a good clean POSIX definition and no user interface
could come in that was not specified by OpenGroup.org. OpenGroup.org is
THE Unix spcification. Documentation (like the Inviolables.md) still
claim that OpenGroup.org is the standards base.
3. Most NuttX users are also Linux users so there has always been
pressure to follow Linux interface definitions (the LSB). After Xiaomi
adopted NuttX, the pressure to adopt the LSB as the specification base
(NSB?) was irresistible.
From my point of view, I am just happy that we can point to
specifications and say, "NuttX is compliant with that" -- whether that
is OpenGroup.org or the LSB.
I have led technical teams for several decades and one thing I have
learned is that everyone does thing differently and as a technical
leader, you have to stand back and simply ask does the propose solution
work? Does the proposed solution meet the need? If yes, then you just
have to accept the solution and let the rest go as personal preference.
Unless you are the BDFL as I was for so many years.
Also, the TERMIOS changes are POSIX and compliant with the LSB,
OpenGroup.org and, I am sure, BSD.
A bunch of caveats.
- Use of the term POSIX compliance OS in regard to any OS that is not
POSIX certified is a trademark violation, but people say this all of the
time.
- http://get.posixcertified.ieee.org/certification_guide.html
- POSIX is a trademark of the IEEE
- Unix is a trademark of OpenGroup.org
- Linux is a trademark of Linux Torvalds
On 3/11/2023 12:08 PM, Tomek CEDRO wrote:
> Please consult Unix manual and source code (for instance any of the BSD).
> Linux was never a reference in Unix world, it was about to mimic Unix, but
> was not even self-compatible. Not a good reference point. See how big mess
> it introduced in current drivers implementation (i.e. drm kms) and their
> propagation to other OS. Is this similar situation here?
>
> Device driver should not provide any additional processing / silent rewrite
> of data, just provide raw data, that is then processed by the application.
> Driver may be configured by ioctl/sysclt, to enable additional / specific
> behavior (i.e. debug, buffer size, latency, etc), so the user always knows
> exactly what the driver does.
>
> I am pro Greg approach, so driver is always simple and generic, everything
> else is up to application. This is the Unix way and you will know how most
> things work, because they all work the same generic standardized way, as
> simple as possible, nothing else happens "in the background unnoticed".
>
> If a specific device needs a specific handling by a generic driver, then
> "quirks" may be used. That explicitly mark "specific" behavior for a given
> device (and only this device).
>
> --
> CeDeROM, SQ7MHZ,http://www.tomek.cedro.info
>
Re: 答复: 答复: 答复: [External Mail]Re: [Breaking change] Echo character by serial driver
Posted by Tomek CEDRO <to...@cedro.info>.
ACK! :-)
--
CeDeROM, SQ7MHZ, http://www.tomek.cedro.info
On Sat, Mar 11, 2023, 23:53 Xiang Xiao <xi...@gmail.com> wrote:
> On Sun, Mar 12, 2023 at 2:09 AM Tomek CEDRO <to...@cedro.info> wrote:
>
> > Please consult Unix manual and source code (for instance any of the BSD).
> > Linux was never a reference in Unix world, it was about to mimic Unix,
> but
> > was not even self-compatible. Not a good reference point. See how big
> mess
> > it introduced in current drivers implementation (i.e. drm kms) and their
> > propagation to other OS. Is this similar situation here?
> >
> >
> This change isn't specific to Linux:
> https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/termios.h.html
>
>
> > Device driver should not provide any additional processing / silent
> rewrite
> > of data, just provide raw data, that is then processed by the
> application.
> > Driver may be configured by ioctl/sysclt, to enable additional / specific
> > behavior (i.e. debug, buffer size, latency, etc), so the user always
> knows
> > exactly what the driver does.
> >
> > I am pro Greg approach, so driver is always simple and generic,
> everything
> > else is up to application. This is the Unix way and you will know how
> most
> > things work, because they all work the same generic standardized way, as
> > simple as possible, nothing else happens "in the background unnoticed".
> >
> > If a specific device needs a specific handling by a generic driver, then
> > "quirks" may be used. That explicitly mark "specific" behavior for a
> given
> > device (and only this device).
> >
> >
> All changes are controlled by ioctl and specified by standard, please read
> the spec:
> https://pubs.opengroup.org/onlinepubs/9699919799/functions/tcsetattr.html
> https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/termios.h.html
>
>
> > --
> > CeDeROM, SQ7MHZ, http://www.tomek.cedro.info
> >
>
Re: 答复: 答复: 答复: [External Mail]Re: [Breaking change] Echo character by serial driver
Posted by Xiang Xiao <xi...@gmail.com>.
On Sun, Mar 12, 2023 at 2:09 AM Tomek CEDRO <to...@cedro.info> wrote:
> Please consult Unix manual and source code (for instance any of the BSD).
> Linux was never a reference in Unix world, it was about to mimic Unix, but
> was not even self-compatible. Not a good reference point. See how big mess
> it introduced in current drivers implementation (i.e. drm kms) and their
> propagation to other OS. Is this similar situation here?
>
>
This change isn't specific to Linux:
https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/termios.h.html
> Device driver should not provide any additional processing / silent rewrite
> of data, just provide raw data, that is then processed by the application.
> Driver may be configured by ioctl/sysclt, to enable additional / specific
> behavior (i.e. debug, buffer size, latency, etc), so the user always knows
> exactly what the driver does.
>
> I am pro Greg approach, so driver is always simple and generic, everything
> else is up to application. This is the Unix way and you will know how most
> things work, because they all work the same generic standardized way, as
> simple as possible, nothing else happens "in the background unnoticed".
>
> If a specific device needs a specific handling by a generic driver, then
> "quirks" may be used. That explicitly mark "specific" behavior for a given
> device (and only this device).
>
>
All changes are controlled by ioctl and specified by standard, please read
the spec:
https://pubs.opengroup.org/onlinepubs/9699919799/functions/tcsetattr.html
https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/termios.h.html
> --
> CeDeROM, SQ7MHZ, http://www.tomek.cedro.info
>
Re: 答复: 答复: 答复: [External Mail]Re: [Breaking change] Echo character by serial driver
Posted by Tomek CEDRO <to...@cedro.info>.
Please consult Unix manual and source code (for instance any of the BSD).
Linux was never a reference in Unix world, it was about to mimic Unix, but
was not even self-compatible. Not a good reference point. See how big mess
it introduced in current drivers implementation (i.e. drm kms) and their
propagation to other OS. Is this similar situation here?
Device driver should not provide any additional processing / silent rewrite
of data, just provide raw data, that is then processed by the application.
Driver may be configured by ioctl/sysclt, to enable additional / specific
behavior (i.e. debug, buffer size, latency, etc), so the user always knows
exactly what the driver does.
I am pro Greg approach, so driver is always simple and generic, everything
else is up to application. This is the Unix way and you will know how most
things work, because they all work the same generic standardized way, as
simple as possible, nothing else happens "in the background unnoticed".
If a specific device needs a specific handling by a generic driver, then
"quirks" may be used. That explicitly mark "specific" behavior for a given
device (and only this device).
--
CeDeROM, SQ7MHZ, http://www.tomek.cedro.info
Re: 答复: 答复: 答复: [External Mail]Re: [Breaking change] Echo character by serial driver
Posted by Gregory Nutt <sp...@gmail.com>.
On 3/10/2023 5:30 PM, Qi3 Huang 黄齐 wrote:
I have to agree that the code is looking better all of the time. Many of
the obvious problems that I saw just a couple of days ago have been
addressed. Many of the things I have complained about in this thread
have been fixed. But there are probably more issues that will show up.
>> I think my biggest concern with the solution is that it > effectively
>> makes it impossible to received any out-of-band data: Any > encoded,
>> context-dependent, escaped data will be echoed back to the > host
>> unconditionally making it impossible to receive cleanly.
> But this behavior can be disabled by termios if needed, and it only
> affect console port by default. And cr/lf convertion also break this.
That is not always true.
* The ECHO behavior can be disabled ONLY if TERMIOS are enabled.
TERIMIOS is now required by POSIX and, further, if TERMIOS is not
system, many features are now broken. Like hiding the password when
logging into NSH.
If isconsole is set and TERMIOS is not, then it will always echo the
input. The CR-LF behavior only depends on isconsole and can't be
changed even if TERMIOS is enabled.
* Also, the trick of disabling ECHO is not a general solution. You
cannot disable TERMIOS for asynchronous, out of band input because
you don't know when that input is going to arrive.
>> We have already seen this in escape sequences sent from the > host
>> appearing appearing on the user console. An awful solution > was
>> adopted: Just remove the escape sequence. That is called >
>> "sweeping the problem under the rug." This will be a serious >
>> limitation that will effect our capability to implement many serial >
>> protocols and bad thing to happen to the OS.
> The current approach could be improved, such as let readline/cle to
> erase the escape sequence on console, what's your suggestion ?
In the case of readline, we know when the ESC sequence is coming.
readline() asked for the position and the host responded. For all
such synchronous input, we could control the ECHO with TERMIOS. But
we cannot for other asynchronous, out-of-band input.
> In general, I think it's a step forward, why we submit such a patch?
>
>
> 1.Telnet before this patch always echo input twice since telnet echo
> character itself by default.
I have never seen that behavior. That bug is probably introduced recently.
答复: 答复: 答复: [External Mail]Re: [Breaking change] Echo character by serial driver
Posted by Qi3 Huang 黄齐 <hu...@xiaomi.com.INVALID>.
> I think my biggest concern with the solution is that it > effectively makes it impossible to received any out-of-band data: Any > encoded, context-dependent, escaped data will be echoed back to the > host unconditionally making it impossible to receive cleanly.
But this behavior can be disabled by termios if needed, and it only affect console port by default. And cr/lf convertion also break this.
> We have already seen this in escape sequences sent from the > host appearing appearing on the user console. An awful solution > was adopted: Just remove the escape sequence. That is called > "sweeping the problem under the rug." This will be a serious > limitation that will effect our capability to implement many serial > protocols and bad thing to happen to the OS.
The current approach could be improved, such as let readline/cle to erase the escape sequence on console, what's your suggestion ?
In general, I think it's a step forward, why we submit such a patch?
1.Telnet before this patch always echo input twice since telnet echo character itself by default.
2.Many application must do echo itself, such as readline/cle/bas. But the bigge problem is, for some software ported from other posix compatible system, they don't do echo by themself, for these applications like WAMR's repl (respacially with REPL) you can't see the input unless you modify the source to adapt it to NuttX.
________________________________
发件人: Qi3 Huang 黄齐
发送时间: 2023年3月11日 0:05:41
收件人: dev@nuttx.apache.org
主题: 答复: 答复: 答复: [External Mail]Re: [Breaking change] Echo character by serial driver
Thanks for point that, so my change will affect them by serial.c since it's in common logic (excatly, is in uart_read()).
________________________________
发件人: Gregory Nutt <sp...@gmail.com>
发送时间: 2023年3月10日 23:58:29
收件人: dev@nuttx.apache.org
主题: Re: 答复: 答复: [External Mail]Re: [Breaking change] Echo character by serial driver
[外部邮件] 此邮件来源于小米公司外部，请谨慎处理。若对邮件安全性存疑，请将邮件转发给misec@xiaomi.com进行反馈
> Emm, I confirmed that one of my test platform (esp32c3), its cdcacm driver implement the uart_ops, but rp2040 seems not.
>
>
> CLE should works since it will override the content from driver ECHO if over serial driver, but readline shouldn't echo input.
>
>
> I'll test more to confirm it.
cdcacm.c is the part that is similar to serial.c. It implements the
serial driver interface and also interfaces with the lower level USB
device driver. cdcacm.c resides at drivers/usbdev/cdcacm.c and it
common for all MCUs. The MCU-specif logic has no idea that there is a
cdcacm "gadget" attached to it.
cdcacm.c implements the common uart_ops that are used by all MCUs to
support CDC/ACM serial.
pl2303.c is another USB serial device. It also implements uart_ops and
has its own TERMIOS implementation.
#/******本邮件及其附件含有小米公司的保密信息，仅限于发送给上面地址中列出的个人或群组。禁止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制、或散发）本邮件中的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并删除本邮件！ This e-mail and its attachments contain confidential information from XIAOMI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it!******/#
答复: 答复: 答复: [External Mail]Re: [Breaking change] Echo character by serial driver
Posted by Qi3 Huang 黄齐 <hu...@xiaomi.com.INVALID>.
Thanks for point that, so my change will affect them by serial.c since it's in common logic (excatly, is in uart_read()).
________________________________
发件人: Gregory Nutt <sp...@gmail.com>
发送时间: 2023年3月10日 23:58:29
收件人: dev@nuttx.apache.org
主题: Re: 答复: 答复: [External Mail]Re: [Breaking change] Echo character by serial driver
[外部邮件] 此邮件来源于小米公司外部，请谨慎处理。若对邮件安全性存疑，请将邮件转发给misec@xiaomi.com进行反馈
> Emm, I confirmed that one of my test platform (esp32c3), its cdcacm driver implement the uart_ops, but rp2040 seems not.
>
>
> CLE should works since it will override the content from driver ECHO if over serial driver, but readline shouldn't echo input.
>
>
> I'll test more to confirm it.
cdcacm.c is the part that is similar to serial.c. It implements the
serial driver interface and also interfaces with the lower level USB
device driver. cdcacm.c resides at drivers/usbdev/cdcacm.c and it
common for all MCUs. The MCU-specif logic has no idea that there is a
cdcacm "gadget" attached to it.
cdcacm.c implements the common uart_ops that are used by all MCUs to
support CDC/ACM serial.
pl2303.c is another USB serial device. It also implements uart_ops and
has its own TERMIOS implementation.
#/******本邮件及其附件含有小米公司的保密信息，仅限于发送给上面地址中列出的个人或群组。禁止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制、或散发）本邮件中的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并删除本邮件！ This e-mail and its attachments contain confidential information from XIAOMI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it!******/#
Re: 答复: 答复: [External Mail]Re: [Breaking change] Echo character by serial driver
Posted by Gregory Nutt <sp...@gmail.com>.
> Emm, I confirmed that one of my test platform (esp32c3), its cdcacm driver implement the uart_ops, but rp2040 seems not.
>
>
> CLE should works since it will override the content from driver ECHO if over serial driver, but readline shouldn't echo input.
>
>
> I'll test more to confirm it.
cdcacm.c is the part that is similar to serial.c. It implements the
serial driver interface and also interfaces with the lower level USB
device driver. cdcacm.c resides at drivers/usbdev/cdcacm.c and it
common for all MCUs. The MCU-specif logic has no idea that there is a
cdcacm "gadget" attached to it.
cdcacm.c implements the common uart_ops that are used by all MCUs to
support CDC/ACM serial.
pl2303.c is another USB serial device. It also implements uart_ops and
has its own TERMIOS implementation.