[uClibc 0006024]: openpty() doesn't support old pseudoterminals

[bugs at busybox.net]

A NOTE has been added to this issue. 
====================================================================== 
http://busybox.net/bugs/view.php?id=6024 
====================================================================== 
Reported By:                michael_d
Assigned To:                uClibc
====================================================================== 
Project:                    uClibc
Issue ID:                   6024
Category:                   Other
Reproducibility:            always
Severity:                   minor
Priority:                   normal
Status:                     assigned
====================================================================== 
Date Submitted:             11-07-2008 14:53 PST
Last Modified:              11-07-2008 16:05 PST
====================================================================== 
Summary:                    openpty() doesn't support old pseudoterminals
Description: 
The openpty() function in libutil fails to work with old style
pseudoterminals (/dev/ttypXX and /dev/ptypXX).  This is because it now
uses the new posix_openpt() function, which presently only tries to use
new style psedoterminals (/dev/ptmx).

I've created a patch to fix this.  It moves the call to the fallback old
terminal routines to within posix_openpt(), instead of directly from the
getpt() wrapper function.

This patch only handles the case where the open flags supplied to
posix_openpt() are O_RDWR.  It would be straightforward to implement
posix_openpt() with other flags for old terminals, but would make the
patch much more complicated.  This patch is simple and suffices to
eliminate the regression from 0.9.29.
====================================================================== 

---------------------------------------------------------------------- 
 bernhardf - 11-07-08 16:05  
---------------------------------------------------------------------- 
I'll think about removing the fallback from posix_openpt some more..
Other than that, i'd suggest:
Index: libc/stdlib/getpt.c
===================================================================
--- libc/stdlib/getpt.c	(revision 23964)
+++ libc/stdlib/getpt.c	(working copy)
@@ -108,6 +108,11 @@ posix_openpt (int flags)
 	    return -1;
 	}
     }
+#if !defined __UNIX98PTY_ONLY__ && defined __UCLIBC_HAS_GETPT__
+  /* If we have no ptmx then ignore flags and use the fallback.  */
+  if (_state & have_no_dev_ptmx)
+    return __bsd_getpt();
+#endif
   return -1;
 }
 libc_hidden_def(posix_openpt)
@@ -126,7 +131,7 @@ getpt (void)
 	return fd;
 }
 
-#if !defined __UNIX98PTY_ONLY__
+#if !defined __UNIX98PTY_ONLY__ && defined __UCLIBC_HAS_GETPT__
 # define PTYNAME1 "pqrstuvwxyzabcde";
 # define PTYNAME2 "0123456789abcdef";
 


Thanks for the heads-up! 

Issue History 
Date Modified   Username       Field                    Change               
====================================================================== 
11-07-08 14:53  michael_d      New Issue                                    
11-07-08 14:53  michael_d      Status                   new => assigned     
11-07-08 14:53  michael_d      Assigned To               => uClibc          
11-07-08 14:53  michael_d      File Added: uClibc-0.9.30-rc3-openptfix.diff     
              
11-07-08 16:05  bernhardf      Note Added: 0015264                          
======================================================================