Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 226893 - Tcl needs a restricted auto_path
Summary: Tcl needs a restricted auto_path
Alias: None
Product: Fedora
Classification: Fedora
Component: tcl
Version: rawhide
Hardware: All
OS: Linux
Target Milestone: ---
Assignee: Marcela Mašláňová
QA Contact:
Depends On: 227200 227951
TreeView+ depends on / blocked
Reported: 2007-02-01 21:34 UTC by Wart
Modified: 2008-01-18 08:18 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Last Closed: 2008-01-18 08:18:36 UTC

Attachments (Terms of Use)
Patch to restrict entries in auto_path (deleted)
2007-02-07 20:11 UTC, Wart
no flags Details | Diff

Description Wart 2007-02-01 21:34:20 UTC
Description of problem:

Tcl searches for packages in 3 directories:
 * /usr/share/tcl8.5
 * /usr/share
 * /usr/lib

The search algorithm involves going into each subdirectory of these three
directories, looking for a file named 'pkgIndex.tcl', and then reading it. 
Since there are quite a few subdirectories in /usr/share and /usr/lib that
aren't tcl-related, this results in a lot of wasted time and effort.

To fix, the default value for 'auto_path' in Tcl needs to be reduced to 2
directories:  /usr/lib{64}/tcl8.x and /usr/share/tcl8.x.  My informal tests have
shown that this can reduce the initial load time for Tcl scripts from 3.7s to 0.2s.

Tcl extension packages in Fedora would also have to be modified to install into
these version-specific tcl directories, instead of the current convention of
/usr/lib, /usr/share.  A 'tcl-sitearch' macro would also be needed in Tcl
package spec files to detect the correct install directory.  This would bring
Tcl more in line with the directory structure used by other scripting languages
(perl, python) in Fedora, and give Tcl applications a boost in startup times.

The extensions can be modified to use the new install directories before the
core Tcl package has changed.  This won't help the startup time, but will
prevent 'package not found' errors if Tcl changes before the extensions.

Version-Release number of selected component (if applicable):
tcl-8.5a5-5.fc7 and all earlier versions

How reproducible:

Steps to Reproduce:
1. Put the following in a file named 'tclpath.tcl' (note that line 2 is
commented out):
#set auto_path [list /usr/share/tcl8.5 /usr/lib/tcl8.5]
package require http

2. Time how long it takes to execute the script:  'time tclsh tclpath.tcl'
3. Reboot to flush the kernel and disk memory caches
3. Uncomment line 2 in tclpath.tcl (set auto_path ...)
4. Time how long it takes to execute the script again.
Actual results:
This takes ~3.7s on my test system for the first run, and ~.2s for the second.

Expected results:
The first run takes just as long as the second.

Additional info:

The timing may vary depending on the state of the kernel's in-memory filesystem
cache and what is in the disk's own buffer.  For more comparable tests, run the
test script as the first command after a reboot.

I haven't poked into the Tcl internals to find the best way to fix the default
package search path, but I'll post a patch if I find a decent way to do it.

Comment 1 Marcela Mašláňová 2007-02-05 14:09:32 UTC
Good point, but I think the upstream should be the most capable to solve it.

Comment 2 Wart 2007-02-06 18:15:24 UTC
I have contacted upstream about this, and their response was expected:  Tcl has
10 years of history of using /usr/lib and /usr/share as package directories, and
if it stops doing so then applications break.

I agree that this is fine for upstream's policy, because they have to deal with
existing installations of extensions, and with extensions installed manually. 
But Fedora is in a position to fix that for our own distribution.  Fedora can
control where Fedora packages are installed, which means we can put all of
Fedora's tcl extensions into %{tcl_sitearch} and %{tcl_sitelib} and achieve this
optimization.  If there are any Fedora applications that are hardcoding
/usr/lib, then we have the ability to fix them.

btw, I have a patch that fixes this, but I want to test it a little more before
attaching it.

Comment 3 Marcela Mašláňová 2007-02-07 10:43:27 UTC
Ok, we can fix it.

Comment 4 Wart 2007-02-07 20:11:03 UTC
Created attachment 147591 [details]
Patch to restrict entries in auto_path

This patch modifies init.tcl to prevent it from adding /usr/lib{64} and
/usr/share onto the package search path.  This patch should not be committed
until bug #227200 has been committed and package maintainers have had a chance
to update their packages to install into the new package directories.  It might
be best to wait for the Fedora Packaging committee to approve the proposed Tcl
packaging guidelines:

Comment 5 Marcela Mašláňová 2008-01-18 08:18:36 UTC
fixed in tcl-1:8.5.0-6

Note You need to log in before you can comment on or make changes to this bug.