the

project

Bugs
MS-Windows problems
Check
Improvements
Caching
Ports and Packages
Version Control
Dependency checking
Documentation
New features
Various

Bugs

• Using both sub/$BDIR/tt.o and $BDIR/sub/tt.o? How about child recipes, the $BDIR is relative to the recipe, how can a parent obtain a result? Problem: Not using the same build directory if a parent builds the same file as a child in a sub directory. Be able to set the location of $BDIR?
  For "../file.o" the "../" is replaced with "_/". Does that work properly? What if the parent sets two variants, does the child take this over? (Bronnikov) Can set the build directory with {var_BDIR = something} attribute, is that sufficient?
  Alternate solution: Put the build directory always in the directory of the source file. Thus "../src/foo.c" gives "../src/build-XYZ/foo.o" instead of "build-XYZ/_/src/foo.o". Avoids subdirectories in the build directory. Explain how to handle read-only source directories, but causes trouble when two recipes compile the source file in different ways when using a different directory instead of a different $BDIR (e.g., one with "-DDEBUG").
• When using "--nobuild": reading a file that was to be produced by a skipped shell command will fail. Make recipe errors non-fatal, continue with the next command.
• aap_shell() doesn't work properly on all systems, esp. Macintosh
• When downloading through ftp and the file doesn't exist, sometimes there is no exception and the resulting file contains "foo: No such file or directory". Read the file to check this?
• When downloading with ftp and the file isn't readable the file ends up the "ls -l" listing of the file. This is a bug in urllib.urlretrieve(), has been reported to the Python bug tracker, ID 635453.
• When the "finally" target fails, dependencies that postponed building until there (by setting a variable) won't be build next time. Add dependency on successful build of "finally" target? Add command to clear out the signature of a target?
• The connection to a rootshell, used for ":asroot" and installing a package, does not always echo stderr, because it is used for stdin, which may not be used bidirectioannly through a pipe. Can this be done with sockets? Requires a safe solution, since it involves a shell with root permissions.

MS-Windows problems

• When using cygwin tools need to use forward slashes, otherwise need to use backslashes. How to handle this?
  - Use backslashes everywhere, allow the use of forward slashes. User must convert names to forward slashes when needed.
  - Add an option to specify what to use. Need to avoid the use of os.path.join()? Default depends on type of shell used?
  - Use a function to override the option, to be used for invoking specific system commands. May use distutils.util...
  - use $/var to change backslashes to slashes, $\var to change slashes to backslashes. Used for cmd.exe or sh.exe on MS-Windows.
  Make a clear description of this in the documentation.
• The return value from os.system() is unreliable. How to get a useful indication if the command failed or not? Ask in a Python maillist? Also use the solution in rectest/test005.py. Also support "tee" if possible. Can use CreateProcess, need to take care of redirection ourselves then. See: http://twistedmatrix.com/pipermail/twisted-python/2002-March/000882.html
• Vim runtime files: need to use the aap.bat file to run A-A-P. Should install it somewhere in $PATH and adjust the path to the Aap directory, like Vim does. On NT we can use ASSOC, FTYPE and PATHEXT to associate a .py file with Python. Use distutils? http://python.org/doc/2.2/dist/simple-example.html
• GUI installer: For MS-Windows use a recipe to install software. Use ideas from NSIS installer? Or make clear that a recipe is not to be used for this?
• In aap_syspath() do a better check if a command will not work for async and MS-Windows, because there is no return value.
• MS-Windows: get system command output with pipes. Check out popen2.py (Steven Knight)
• Support MSVC, use distutils.msvccompiler.

Check

• Are there still external commands that use single quotes around arguments? This doesn't work for MS-Windows.
• Try installing wxPython on MS-Windows and run Agide there. Find out what doesn't work.
• When copying variables from a recdict, copy $var from the same recdict the value was copied from, not any other.
• A child recipe with an "origin" attribute and a ":recipe" command inside it should only be fetched once.

Improvements

• When "aap test" can't find unittest.py, use a recipe do download it and put it in the unittest directory.
• Read a recipe always as UTF-8. (Steve Howe) Does this only need to be specified in the documentation? Detect and ignore a BOM at the start of the recipe?
• Add a command-line argument to define the preferred tools? E.g. to use MingW when MSVC can also be found.
• Support a condition after variant values: 'much compiler == "gcc"'. Especially useful with the "*" value, then also allow it multiple times.
• Escaping special characters in shell commands depends on the shell used. Adjust dictlist2shellstr()
• "aap distclean" should remove the signature files and build directories. What of "AAPDIR" can be removed? Or use "aapclean" target?
• ":do" command in examples: It's unclear what the target is. (CWI) Add target in examples? Pass the target name in another way?
• When publishing with "scp://" directories are created automatically. Also do this for other methods, e.g., "file:/".
• When a shell command changes a file that's not mentioned as a target and its signature was already computed, the change will not be noticed, because the signature is cached.
• Give a clearer error when multiple rules match, mention the rules (and that the source file doesn't exist). (Bronnikov)
• Make sufreplace() work better: don't replace directory names.
• ":copy -i" should check existence of remote files and directories (how?)
• Problem with configure.in and configure: How do we know configure needs to be generated or not? Put checksum of configure.in inside the generated configure? Or is "mysign" good enough?

  	auto/configure {signline = "# MD5=%s"}: configure.in
  		:sys autoconf

• Give a warning for an assignment to a variable that is set automatically: "source", "target", "async", etc.
• After checking out the "Exec" module, renaming "Exec/" to "aap/", doing "aap fetch" in "aap/" doesn't work. Detect this situation and tell CVS to checkout the "Exec" module into directory "aap"? (Antonio Colombo)
• Print "Stop." when building stops because of an error.
• Use "E123" and "W123" for all messages, so that they can still be recognized after translation.
• For non-Posix, use os.path.normcase() on Node names.
• Shell commands are executed several at a time, it's not clear that an error message applies to any one of them. Better error message?
• Add something to cleanup items from the AAP/sign file for old targets. E.g., upload.aap creates a zip file that includes the version number. Idea from Maak: Use the signatures to know which files were produced. These are candidates for the "clean" or "distclean" targets. Idea from Maak: provide a function to get a list of produced files. The list can be filtered for various purposes. Should also be possible to obtain produced files for a specific target: list all files used to produce "foo.exe".
• In default.aap, turn the preproc check for GCC to a Python function, so that it can be used elsewhere: res = try_preproc(globals(), string, "-Idir") Also try_compile(), try_link() and try_run().
• For publishing files, have some way to only publish when the destination file is different. Useful when downloading is fast and uploading is slow and avoids having to enter a password (for remote file or to become root).
• Using ":copy" with scp, support an attribute to use "-C" or not (compression). Also explain use of ~/.ssh/config.
• Allow the scheme of a URL to be upper case.
• For the issue tracker: provide a way to fill in some of the fields of a form on a web page. E.g., for the version number and the name of the user.

Caching

• When downloading distributed archives, it doesn't matter where they were downloaded from. Only use the basename when looking up the file in the cache.
• For the cache: re-use the old file name when downloading the same file again? If not, is the old file really deleted?
• Remove the outdating of cached files? Makes using remote files more predictable: only update files when fetching is wanted. No need to check timestamps, a cached file is always considered valid.
• Write code to cleanup caches, delete old files, keep size below a limit.
• Check that the directory of the main recipe and children is writable. Give a clear error (e.g., when AAPDIR/log can't be written).

Ports

• Vim port: verify "clean" and "distclean" targets delete what should be deleted.
• When a package can be installed with "aap --install abc" then "aap --uninstall abc" should be possible to uninstall it again.
• Vim port install: Get "gzip: stdout: Broken pipe" but it does work. Appears to happen when output of pkg_add is redirected. Can't reproduce it with shell commands though.
• config.arg is for configure arguments. Write a config.txt file for Vim as an example.
• How about supporting other recipe variable settings and environment variables for configure?
• Name for a port recipe: vim.aap. Creates "vim" directory for the files. Avoids another directory level. (Benji Fisher)
• For dependencies, allow a method to specify the package to install.
• Vim example: depending on GTK or Motif for DEPEND_RUN doesn't work: it depends on what configure decided to use. What is a nice way to handle this?
• Implement CVSTAG: tag to use for CVS download. Also for branches. (Vince Negri) Should also work in non-port recipe. Use "cvstag" attribute?
• Check a checksum right after downloading, if it doesn't match continue downloading with the next site. Helps for broken mirrors (esp. empty file).
• When a dependency is not met, allow specifying what package to install (search path?). DEPENDS = vim-* {install = vim-big}
• When following dependencies, use them from the port recipe, but may install the binary package to avoid building (ignoring its dependencies).
• Implement checking if missing dependencies can be installed.
• Implement automatic install for missing dependencies.
• When patching don't do anything interactive, fail instead of asking.
• For patching give a summary of the results, not the whole verbose output. Grep for "failed".
• If patching fails, should delete the unpacked files, this needs to be done again: trigger "clean". Or do "clean" before unpacking?
• When patching a file like "configure.in" in Vim, should first back out aap-specific changes, patch, then re-apply the changes.
• Using existing package system of the computer. Make a generic interface to various packaging systems. Check what RPM requires.
• Pass on to package system: Install for tryout, install for real.
• Remark: Package system should allow backup before install, restore old version, delete old version.
• When a program requires a HTML file viewer, it wants to specify this dependency, without knowing which packages offer an HTML viewer. Make a list of features that packages offer/require? Portage has something like this. Make mapping of features to packages available somewhere on the net.

Version Control

• Support using a specific tag/branch. Add "cvstag" attribute?
  When a recipe specifies a different branch, update to the other one with: cvs -q update -d -r <branch-name> (Bob Hiestand) Or use "checkout", would do the same thing.
• Support other version control systems. For example SubVersion (not at version 1.0 yet) or Perforce (free up to two users).
• Starting a new CVS module could be automated. It is documented in the user-cvs chapter: copy the files to a temp directory and do "cvs start". Requires asking the user for the name of the module and project. Might even init the CVS repository automatically. And install cvs first!
• Support branching somehow, because it's not easy to do.
  aap branch NAME=xyz
  aap merge FROM=xyz INTO=main
  Read: Working with branches and survive
  Suggestions from Erik Christiansen:

        aap tag  NAME=foobar		(already implemented)
        aap tag-delete NAME=foobar	(not very useful)
        aap tag-float NAME=foobar	(useful, should be added)

  Docs: Good method to use branches: one branch per bugfix (Eli Carter, Nov 15).
• Patch generation: "aap diff"; Generate diff with the version that was checked out.
• Add CVS password somehow, example from Greg Steuck, Oct 3 2002.
• Take a look at tkCVS and cervisia: GUI frontends to CVS.
• CVS: When adding, automatically detect a file is binary (using filetype).
• CVS can (and will) merge in changes the user made himself. How about other version control systems? What if the local changes are to be abandoned. (difference between checkout and update?)
• CVS: Check for this error message: "cvs [checkout aborted]: connect to <servername> failed: Connection refused"

Dependency checking

• When the directory of a source is not writable, where to put the auto-dependency file? Relative to the directory of the recipe?
• For compilers that don't generate dependencies like gcc, at least use the preprocessor with "cc -E" and scan the output for lines that indicate included header files.
• When invoking an automatic dependency checker, the arguments for the build command may depend on how the file is used (different CFLAGS used).
• Computing MD5 checksums can take a very long time. Add alternative to only compute the checksum when the timestamp of the file has changed: {check = time md5}.
• When wanting to use "newer" check for everything (like make), how to apply this to autodepend files?
• Option to check for changed system header files and libraries to trigger rebuilding (like SCons). May need to rerun configure.
• Allow defining a rule based on filetype. At least for the target. Doing it for the source has the problem of having to guess what the file name of the source file is.
  Possibly allow using all the defined actions as rules. Has the same problem of guessing the source file name.

Documentation

• ":sys cd dir" followed by a command that should work in "dir" doesn't work. Mention that ":cd dir" is to be used.
• On the web site: Add a search engine for the manual.
• Move some things from tutorial to user manual, keep the tutorial simple.
• Find simple method for making cross-references (predefine certain words).
• Improve the layout, remove the plain-text parts.
• Move the list of Python functions from the user to the reference manual.
• Read manual for Cook, take over examples?
• Formal specification of recipe format.
• Add chapter on security: using .netrc for password, recommend using rcp instead of ftp for uploading.
• Generate list of patches: 6.1.001, 6.1.002, ... 6.1.171 in a simple way? Note the extra zeros. Explain how to do it with range().
• Explain how Make-style variable changes (e.g., ${VAR:C/abc/def/}) are done in Python.

New features

• support parenthesis: (foo bar) {attr = val}
• Add ":cd" command. (Patch from Steve Howe)
  Always chdir after executing build commands, in case ":cd" is used?
  Correct docs, popd doesn't take a name argument. Make $source and $target full path names, so that they remain valid after ":cd".
  When shortening file names for messages, do this relative to the dir of the toplevel directory.
• Add command line argument "--check=newer". Overrules the checks specified in the recipe and uses timestamps like "make". Must still compute the signatures as before and stores them for updated targets, so that a following invocation without a "--check" argument doesn't build them again.
  Also add the $_DEFCHECK variable: default check to be used. When not set "md5" is used.
• When using "scp://" first check if the "scp" program is available. If not, offer the user to download and install it.
• Alternative upload/download method: Use automatic version numbering, put the version number in each file name. Does not require a CVS server, easy to maintain, takes more disk space.
  - make a link from the generic file name to the file with the latest version.
  - requires a file locally to remember which version was last published for each file.
  - tagging means generating a recipe for the current set of files. Include MD5 signatures like ":mkdownload" to detect which files are outdated.
• Make publishing and committing more similar. When publishing, delete files that are no longer published. Internally use an interface to each publish method. One of the functions is used to list the files currently present, so that can be detected which ones are to be deleted.
• Add ":import" command: like ":include" but don't include the same recipe twice. Useful for variants.
• Support searching for a recipe upwards in a directory tree: ":include .../settings.aap". Useful for project-wide settings that are to be used in subdirectories.
• After an error while building a target, when the -k option is used, continue with targets that don't depend on the one with an error. This is only partly implemented yet.
• Add command ":symlink". Use {force} to skip when the correct link already exists.
• Add SCP variable: "scp -C" by default, can be changed by user. More generic: allow the user to redefine how a protocol is handled.
• "curl" and/or "fetch" could be used to download files.
• Add a FETCH variable: used when there is no fetch attribute for a file that isn't present.
• Add more options to the ":tree" command: select certain filetype, select files/dirs with a certain mode (e.g., executable), etc.
• Add a grep() function. First argument: re pattern. Second argument: string variable (list of file names) or list of file names (from glob()). Returns list of files with matches. Optional "reverse" argument: list files without a match.

  	@if grep("config.mk", "auto/configure"):
  	FILES = `grep("version 4", glob("*.c"))`
  	:print Not updated: `grep("updated", str2list(SOURCE), reverse = 1)`

• Idea: remote signatures. Get file that lists the signatures of files in that directory (or directory tree). Use this to decide what to download. Advantage: Only need to download one file to check for changed files reliably. Use an attribute on the "fetch" attribute to specify the file:
  {fetch = http://foo.org/%file% {signlist = http://foo.org/signatures}}
  Is this still needed next to ":mkdownload"?
• Add Java support. Problem: produces multiple .class files from one .java (nested and anonymous classes). Dependencies are hard to figure out. Can leave this to the Java compiler, unless the Java file is generated (using preprocessor).
• Support hooks for user defined signatures. ":action sign"
• In all places where a filetype is used: when filetype isn't found, try using filetype_root().
• In build commands it might be useful to know which sources were outdated. E.g., to delete the configure cache when configargs was outdated. Python function "aap_outdated(target, source)"?
• Support Aap argument "-d flags" for debugging specific things (see make manpage).
• When using $(source[0:3]) use the first three items. $(source[:-1]) all but the last one.
• Add a command to mark a node as "up to date", so that it isn't rebuild (and the files it depends on are not rebuild).
• Implement aap_var2list() and aap_var2dictlist().
• Implement aap_has_target(targetname): check if a target exists.
• Implement aap_was_set(varname): check if variable has been set.
• Implement deleting a remote file.
• Implement a nice interactive method (window for Agide) to let a user select features. Should at least work for autoconf scripts. Store the results in config.arg. (See use case "Install an application from sources").
• Make "skip" attribute of a rule accept a file pattern: "test/*"
• - Automatically uncompress a downloaded file (also a recipe) when it is recognized as a compressed file and the local file name is not compressed.

  	:child unix.aap {fetch = ftp://ni.org/unix.aap.gz}

  Also to extract a file from an archive:

  	:child pc.aap {fetch = ftp://ni.org/pc.zip//main.aap}

  Especially for Java jar archives.
• Allow the user to define a command like ":program" to build a file from several sources. Something like:

  	:define foo
  	    @for item in $source:
  	        :do compile $item
  	    :do build $target $interm
  Invoke this with ":build foo myfoo : source" would work.
  

• Add command to create zip file: ":zip zipfile comment name-list".
• Add command to unpack zip file: ":unzip zipfile [options]".
• Add command to apply a patch: ":patch patchfile [targetfile]".
• Automatic function prototype system for C:
  - Put each prototype in a separate file. Regenerate prototypes when the C file has changed. Make argument names uniform, a change in the name is not a dependency.
  - For each C file include a "file.proto.h" file, which is generated to contain an #include line for each required prototype.
  - Dependencies will automatically recompile a C file in which a function is used for which the prototype changed.
• Push system: Somehow trigger a remote system to run "aap -R" on an existing recipe. Can this be done with rpc? Or use ssh:

  	:sys ssh -f user@machine aap --fetch /dir/doit.aap

Various ideas

• How to give a warning for an attribute that isn't used? E.g., "{time}" that should be "{check = time}"
• When building a large tree, errors are found faster if first building files that don't exist at all before checking dependencies.
• How about cross-compiling?
• Is there a way to have the kind of dependency checking depend on the rule used? E.g. for foo.c -> foo.o ignore comments, for foo.c -> foo.doc do use comments. At least allow specifying the kind of dependency checking used.
• For "aap comment": remember name of recipe where target was specified.
• Lint-like tool for recipes? Check for unused variables and attributes. Or a verbose option to the Aap command?
• Debugging: Single-stepping through a recipe.
• Some form of locking to avoid two people execute recipes in the same directory? Avoids aap/sign file gets messed up.