Kara-Moon Forum
March 29, 2024, 06:26:24 AM *
Welcome, Guest. Please login or register.

Login with username, password and session length
News: You can go back to the main site here: Kara-Moon site
 
   Home   Help Search Login Register  
Pages: [1] 2
  Print  
Author Topic: Non-standard Installs  (Read 6044 times)
bvdp
Kara-Moon Master
****
Posts: 1436


WWW
« on: November 22, 2020, 06:32:10 PM »

I _think_ I may have a solution for non-standard installations. A simple change to the main calling script, mma.py, now forces a check for the env variable MMA_MMADIR. So, you can set it like this:

   MMA_MMADIR=./MMA-2002 mma test

or, like any env variable in a .profile or .bashrc script.

A bit of caution:

   - we're assuming MMA-2002 has a MMA directory
   - the lib, plugins and include directories also have to be present. You can do this via links in the directory or by setting the appropriate env variables.
   - we are expanding the variable to complete pathname. I'm not sure if this is "good" or "bad".

Comments?

* mma-py.txt (3.05 KB - downloaded 151 times.)
Logged

My online life: http://www.mellowood.ca
sciurius
Sr. Member
****
Posts: 443



« Reply #1 on: November 23, 2020, 10:08:51 AM »

The goal of non-standard installs (funny name, since actually these should be the standard installs) is to be able to impose local material on top of the installed MMA.

This is already possible for libs, plugins and includes via the environment variables MMA_LIBPATH, MMA_PLUGPATH and MMA_INCPATH (see footnote).

For the MMA modules it would be best to rely on Python's module searching, To get the best out of both worlds I tried some experiments and got good results with this:

Evironment variable MMA_HOME can be used to designate the place where the MMA modules (and default libs, plugins and includes) are.
If environment variable is not set, the current approach of searching an appropriate dir from a predefined list of directories.

The relevant code of mma.py becomes:

Code:
MMAdir = environ.get('MMA_HOME')
if not MMAdir:
    for d in dirlist:
        moddir = os.path.join(d, 'MMA')
        if os.path.isdir(moddir):
            if not d in sys.path:
                sys.path.insert(0, d)
                MMAdir = d
                break

try:
    import MMA.main
except ModuleNotFoundError as e:
    print("Unable to find the modules needed for MMA. Please check your installation.")
    sys.exit(-1)

In my environment, I can now set up

Code:
export  MMA_LIBPATH=$HOME/lib/mma/grooves
export MMA_PLUGPATH=$HOME/lib/mma/plugins
export  MMA_INCPATH=$HOME/lib/mma
export     MMA_HOME=/my/nonstandard/location/for/mma
export   PYTHONPATH=$HOME/lib/mma:$MMA_HOME

This allows me to run mma and have it find all standard stuff at the nonstandard locaton, simultaneously being able to add/override all lib, plugins. includes and modules in my private $HOME/lib/mma library.

Does that sound sensible?

Footnote: Note that these are comma-separated, instead of the customary (semi-)colon -- it is too late to change?
Logged
bvdp
Kara-Moon Master
****
Posts: 1436


WWW
« Reply #2 on: November 23, 2020, 10:09:35 PM »

Okay, spent some more time on this.

I've adapted the name MMA_HOME. Probably more clear.

Commas or semi-colons? Hmmm, I've always used commas. I was unable to find anything definitive on the web just now ... seems to be more an application thing than anything else. I suppose we could have a MMA_ENV_LIST_SEPARATOR variable to keep everyone happy ... but that's getting silly. Besides, in most cases I think that only one path will be used, so I'm going to leave it for now.

Note that in all the different versions setting the various LIB paths has been working just fine. Using an env variable to set MMA_LIBPATH, etc has nothing to do with the setting of MMAdir since all those pathnames are expanded to an absolute path. Side question: doesn't this really work better by setting the various paths in a rc file rather than as a env var?

What we are trying to do here in mma.py is to make robust system for finding the main mma tree, no matter how convoluted the installation is.

So, here's the new "logic":

   - if no env variable found we create a path list consisting of some system-specific locations (have a look at the source) depending on the system,
   - the system path (where python expects stuff) is appended to the list created above. So, if MMA is not found in one of the system-specific locations we default out to the system install.

   - finally, if (and only if) a env variable MMA_HOME has been set we wipe out all the stuff above and look only in the passed location. If you have specified a location for MMA and it doesn't exist or is not reachable you WANT an error ... you don't want to blindly continue in an unknown location.

Oh, thanks for reminding me that except/pass is my friend Smiley
 
New mma.py attached.

* mma.py.txt (3.33 KB - downloaded 157 times.)
Logged

My online life: http://www.mellowood.ca
sciurius
Sr. Member
****
Posts: 443



« Reply #3 on: November 24, 2020, 11:14:57 AM »

Close, but no cigar  Smiley

By explicitly prepending the MMAdir to sys.path you defeat the setting of the PYTHONPATH environment variable. That's why in my proposal I do not touch sys.path if MMA_HOME is set.

Also:
Code:
        try:
            import MMA.main
        except:
            print("Found MMA directory in '%s'. Unable to execute MMA.main.py." % MMAdir)
        sys.exit(-1)

This will always terminate with sys.exit(-1), I'm quite sure that is not what you intended.

Quote
Oh, thanks for reminding me that except/pass is my friend
You're welcome... But remember you already found out (the hard way) that you should never catch arbitrary exceptions...
Logged
bvdp
Kara-Moon Master
****
Posts: 1436


WWW
« Reply #4 on: November 25, 2020, 02:10:54 AM »

Close, but no cigar  Smiley

By explicitly prepending the MMAdir to sys.path you defeat the setting of the PYTHONPATH environment variable. That's why in my proposal I do not touch sys.path if MMA_HOME is set.

Olay, let me digest that. Not sure how the python running mma finds anything this way. But, it does make sense. Let me look tommorrow.

Quote
Also:
Code:
        try:
            import MMA.main
        except:
            print("Found MMA directory in '%s'. Unable to execute MMA.main.py." % MMAdir)
        sys.exit(-1)

This will always terminate with sys.exit(-1), I'm quite sure that is not what you intended.

I think this is almost correct ... however I  do need to change the except to

    except ImportError:

So that if there is an error in the script it doesn't think it should fall though to the bottom.
Logged

My online life: http://www.mellowood.ca
sciurius
Sr. Member
****
Posts: 443



« Reply #5 on: November 25, 2020, 07:24:34 AM »

Quote
I think this is almost correct ... however I  do need to change the except to

    except ImportError:

So that if there is an error in the script it doesn't think it should fall though to the bottom.

Yes, but more important is to shift the sys.exit(-1) to the right so it becomes part of the except handling. As it is now when MMA.main completes normally, it will always sys.exit(-1) and I don't think that is desired.
Logged
sciurius
Sr. Member
****
Posts: 443



« Reply #6 on: November 25, 2020, 12:27:07 PM »

There's another python surprise getting in the (well, my) way...

Whatever value you set PYTHONPATH to, python always prepends the script directory. I have to rethink....
Logged
bvdp
Kara-Moon Master
****
Posts: 1436


WWW
« Reply #7 on: November 25, 2020, 06:46:47 PM »

Here's yet another version of mma.py.

I have fixed the sys.exit() issue by creating 2 of them. One for success and another for failure Smiley And, don't ask where I got the -1 value from ... some kind of mind-fart no doubt.

As far as PYTHONPATH goes ... that's got nothing to do with MMA. Python will process that before running mma.py. I've never had occasion to use that myself. I read about a number of people using virtual environment for python and that might be a better way to run?

It is doing better failure reporting if MMA_HOME doesn't work out as well.

* mma.py.txt (3.52 KB - downloaded 160 times.)
Logged

My online life: http://www.mellowood.ca
bvdp
Kara-Moon Master
****
Posts: 1436


WWW
« Reply #8 on: November 25, 2020, 09:37:13 PM »

More on the path separator ... in python there is a constant os.pathsep. In unix (linux and I'm assuming mac) it is set to ":" and in windows ";".

We use comma separated lists right now in MMA when using spaces doesn't work. Which is, probably, why I used a ',' in the env stuff. However, simple enough to change that for the env stuff like LIB_PATH, MMA_HOME, etc.

Comment?
Logged

My online life: http://www.mellowood.ca
sciurius
Sr. Member
****
Posts: 443



« Reply #9 on: November 26, 2020, 09:43:53 AM »

in python there is a constant os.pathsep.
Good!

Quote
We use comma separated lists right now in MMA when using spaces doesn't work. Which is, probably, why I used a ',' in the env stuff. However, simple enough to change that for the env stuff like LIB_PATH, MMA_HOME, etc.
It sounds definitely like a good idea to use os.pathsep in the environment variables. It makes setting these consistent with the rest of the system.

From the perspective of my workflows I do not see a real use for SetLibPath and friends now we have the environment variables. MMA should be about making music, not system path names Smiley.
Logged
sciurius
Sr. Member
****
Posts: 443



« Reply #10 on: November 26, 2020, 01:12:08 PM »

I went back to the drawingboard one more time and my current opinion is that there is only one 'right' approach: make MMA installable using pip.
See e.g. https://packaging.python.org/tutorials/packaging-projects/ .

This involves a bit more than tweaking some paths but should make install and run sufficiently robust. A pip-style source distribution is also ready to be run directly from source.
« Last Edit: November 26, 2020, 01:15:58 PM by sciurius » Logged
sciurius
Sr. Member
****
Posts: 443



« Reply #11 on: December 04, 2020, 10:21:33 PM »

Warning -- Work In Progress!

Attached are two scripts.

setup.py is a standard python installer script, to be used with the 'pip' tool.

To use it, a few mods to the mma sources are needed: Directories egs includes lib plugins util must be moved to below the MMA directory, and
a new directory cli must be created and the attached 'mma' script must be put there.

In commands:

Code:
tar xf mma-devl.20.02f.tar.gz
cd mma-bin-20.02f
mv egs includes lib plugins util MMA
cp ... setup.py
mkdir cli
cp ... cli/mma

For a user mode install, type
Code:
pip install --user .

For a system install, type
Code:
sudo pip install .

This should work on Linux, MacOs and Windows.

Thoughts?

* setup.py.txt (1.33 KB - downloaded 153 times.)
* cli.mma.txt (0.94 KB - downloaded 159 times.)
Logged
bvdp
Kara-Moon Master
****
Posts: 1436


WWW
« Reply #12 on: December 04, 2020, 11:23:08 PM »

Without running code ... does this mean that the modules like chords.py, main.py, etc are sharing the same dir as lib, plugins, etc? That doesn't excite me.

I assume that the calling program, mma.py, will live outside of this directory? Or will it work there or in a bin directory somewhere else?

One thing that has always bothered me (not a big bother, but it's a bit ugly) is my decision many years ago to use MMA for the modules ... at the time I hadn't thought thorough the case issues this could present in windows/mac. I guess it works okay, but I'm wondering if that should be looked at?

I am planning to release the current stuff as 20.12 in the next week. So, we should look at packaging for 2021?
Logged

My online life: http://www.mellowood.ca
sciurius
Sr. Member
****
Posts: 443



« Reply #13 on: December 05, 2020, 09:42:02 AM »

This is what I get:
Code:
/usr/lib/python3.7/site-packages/MMA/after.py
/usr/lib/python3.7/site-packages/MMA/alloc.py
...
/usr/lib/python3.7/site-packages/MMA/includes/...
...
/usr/lib/python3.7/site-packages/MMA/lib/...
...

This is a convention used by most python packages.

I'm going to move the egs and util directories elsewhere (/usr/share/mma/egs and /usr/share/mma/util) since their contents doesn't need to be installed next to the mma code.

The cli/mma program is a new script that will be the installed as '/usr/bin/mma'. The current 'mma.py' script will continue to work in an uninstalled mma file tree.

This is a work in progress so no reason to hurry it in 20.12.

Nice things I found so far: It is trivial to perform a local install (pip install --user) and test everything. pip uninstall will nicely clean up everything.

Imagine all users can install/upgrade mma by just typing 'pip install mma'... No downloads, unpacks, ...
Logged
sciurius
Sr. Member
****
Posts: 443



« Reply #14 on: December 05, 2020, 02:49:33 PM »

Hmm. It seems that Hugo Neves de Carvalho <hugonvsc@gmail.com> already provides a good working pip-installable MMA. Maybe it is time to join efforts?

Code:
Metadata-Version: 2.1
Name: mma
Version: 1.5.20.2
Summary: MMA - Music Midi Accompaniment available as a pip module
Home-page: https://github.com/hugonxc/mma_pip
Author: Hugo Neves de Carvalho
Author-email: hugonvsc@gmail.com
License: UNKNOWN
Platform: UNKNOWN
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: OS Independent
Requires-Python: >=3.6
Description-Content-Type: text/markdown

# MMA—Musical MIDI Accompaniment

This is a fork from [MMA](https://www.mellowood.ca/mma/) to make it available as a pip module.
The current version of MMA in this project is 20.02, which can be different from the main repository.
Logged
Pages: [1] 2
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.21 | SMF © 2015, Simple Machines Valid XHTML 1.0! Valid CSS!
Page created in 0.053 seconds with 20 queries.