Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The word 'select' causes 2d arrays to fail #790

Closed
TristanJamesBall opened this issue Oct 20, 2024 · 9 comments
Closed

The word 'select' causes 2d arrays to fail #790

TristanJamesBall opened this issue Oct 20, 2024 · 9 comments
Labels
bug Something is not working

Comments

@TristanJamesBall
Copy link

TristanJamesBall commented Oct 20, 2024

# typeset -a a1=( (demo)  ("select") (col) (from) (table)  )
# typeset -p a1
typeset -a a1=((demo) (select) (col) (from) (table) )

# eval ${ typeset -p a1; }
/bin/ksh: eval: syntax error: `select' unexpected

# typeset -a a1=( (line0) ( select from ) )
/bin/ksh: syntax error: `select' unexpected

Seems to be specific to the word "select", and it only occurs when it appears as the only element of the array dimenion, where that dimension >0

For example, this works fine:

typeset -a a1=( select col from table )
eval ${ typeset -p a1 ;}

Quoting the "Select" appears to work around it, but typeset -p doesn't quote array elements unless it has too, so you can't eval the output typeset -p

Occurs on both my dev build from here:
Version JM 93u+m/1.1.0-alpha+1c4b3ae6/MOD 2023-06-14
And the distro version:
Version AJM 93u+m/1.0.8 2024-01-01

@McDutchie McDutchie added the bug Something is not working label Oct 20, 2024
@McDutchie
Copy link

Thanks for the report. This is also reproducible on 93u+ 2012-08-01 and on ksh2020.

@McDutchie
Copy link

The problem is that reserved words are incorrectly recognised when assign() is called recursively to parse multidimensional array assignments. Please try this patch:

diff --git a/src/cmd/ksh93/sh/parse.c b/src/cmd/ksh93/sh/parse.c
index 1f88e424d..58dbea2f1 100644
--- a/src/cmd/ksh93/sh/parse.c
+++ b/src/cmd/ksh93/sh/parse.c
@@ -1053,7 +1053,10 @@ static struct argnod *assign(Lex_t *lexp, struct argnod *ap, int type)
 			ar = stkfreeze(sh.stk,1);
 			ar->argnxt.ap = 0;
 			if(!aq)
+			{
+				lexp->noreserv = 1;
 				ar = assign(lexp,ar,0);
+			}
 			ar->argflag |= ARG_MESSAGE;
 			*settail = ar;
 			settail = &(ar->argnxt.ap);

@McDutchie
Copy link

That patch is not quite right. It fixes the reproducer but not this:

$ arch/*/bin/ksh -c 'typeset -a a1=((demo) (select) (col) (from) (table) ); typeset -p a1'
typeset -a a1=((demo) (select) (col) (from) (table) )
$ arch/*/bin/ksh -c 'typeset -a a1=((demo) select (col) (from) (table) ); typeset -p a1'  
arch/darwin.arm64-64/bin/ksh: syntax error at line 1: `select' unexpected

Patch v2:

diff --git a/src/cmd/ksh93/sh/parse.c b/src/cmd/ksh93/sh/parse.c
index 1f88e424d..71a364def 100644
--- a/src/cmd/ksh93/sh/parse.c
+++ b/src/cmd/ksh93/sh/parse.c
@@ -1053,7 +1053,12 @@ static struct argnod *assign(Lex_t *lexp, struct argnod *ap, int type)
 			ar = stkfreeze(sh.stk,1);
 			ar->argnxt.ap = 0;
 			if(!aq)
+			{
 				ar = assign(lexp,ar,0);
+				/* since noreserv is reset to 0 below, we have to restore it after a recursive call */
+				if(array)
+					lexp->noreserv = 1;
+			}
 			ar->argflag |= ARG_MESSAGE;
 			*settail = ar;
 			settail = &(ar->argnxt.ap);

@McDutchie
Copy link

Another approach, which may be safer, is to save and restore the value:

diff --git a/src/cmd/ksh93/sh/parse.c b/src/cmd/ksh93/sh/parse.c
index 1f88e424d..cc30ae7f2 100644
--- a/src/cmd/ksh93/sh/parse.c
+++ b/src/cmd/ksh93/sh/parse.c
@@ -1053,7 +1053,12 @@ static struct argnod *assign(Lex_t *lexp, struct argnod *ap, int type)
 			ar = stkfreeze(sh.stk,1);
 			ar->argnxt.ap = 0;
 			if(!aq)
+			{
+				/* since noreserv is reset to 0 below, we have to restore it after a recursive call */
+				char n = lexp->noreserv;
 				ar = assign(lexp,ar,0);
+				lexp->noreserv = n;
+			}
 			ar->argflag |= ARG_MESSAGE;
 			*settail = ar;
 			settail = &(ar->argnxt.ap);

@McDutchie
Copy link

It's still not right. It should also work without the typeset -a, when the array attribute is assigned implicitly, and it doesn't.

$ arch/*/bin/ksh -c 'my_array=((somevalue) select) && typeset -p my_array'
arch/darwin.arm64-64/bin/ksh: syntax error at line 1: `select' unexpected

@McDutchie
Copy link

This fixes it with or without the explicit typeset -a.

Patch v4:

diff --git a/src/cmd/ksh93/sh/parse.c b/src/cmd/ksh93/sh/parse.c
index 1f88e424d..5e2f9cc0e 100644
--- a/src/cmd/ksh93/sh/parse.c
+++ b/src/cmd/ksh93/sh/parse.c
@@ -1053,7 +1053,11 @@ static struct argnod *assign(Lex_t *lexp, struct argnod *ap, int type)
 			ar = stkfreeze(sh.stk,1);
 			ar->argnxt.ap = 0;
 			if(!aq)
+			{
 				ar = assign(lexp,ar,0);
+				/* since noreserv is reset to 0 below, we have set it again after a recursive call */
+				lexp->noreserv = 1;
+			}
 			ar->argflag |= ARG_MESSAGE;
 			*settail = ar;
 			settail = &(ar->argnxt.ap);

McDutchie added a commit that referenced this issue Oct 23, 2024
Example reproducers:
  $ my_array=((somevalue) (select)) && typeset -p my_array
  -ksh: syntax error: `select' unexpected
  $ my_array=((somevalue) select) && typeset -p my_array
  -ksh: syntax error: `select' unexpected

Analysis: The problem is in the assign() function in parse.c. From
the second value element on, reserved words/keywords are wrongly
recognised when assign() is called recursively to parse
multidimensional array assignments, because the lexer flag
'noreserv' is reset to zero in the course of the assign() call.

src/cmd/ksh93/sh/parse.c: assign():
- After recursively calling assign() to deal with nested
  parentheses, set noreserv to back to 1 to re-disable the
  recognition of reserved words by the lexer.

Thanks to @TristanJamesBall for the report.
Resolves: #790
@TristanJamesBall
Copy link
Author

This is fantastic, thankyou for such a quick turnaround on the fix!

@TristanJamesBall
Copy link
Author

Unfortunately though, there still seems to be an issue - although triggered by the word 'export' not 'select'

typeset -a arr=( (a b c) (export demo array) )
typeset -p arr
typeset -a arr=((a b c) ())

Do you want a new issue raised?

@McDutchie
Copy link

Yes please, that's a different issue, not related to reserved words.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something is not working
Projects
None yet
Development

No branches or pull requests

2 participants