Kara-Moon Forum
March 28, 2024, 09:10:42 PM *
Welcome, Guest. Please login or register.

Login with username, password and session length
News: You can go back to the main site here: Kara-Moon site
 
   Home   Help Search Login Register  
Pages: [1] 2
  Print  
Author Topic: Python 3  (Read 20656 times)
bvdp
Kara-Moon Master
****
Posts: 1436


WWW
« on: December 31, 2013, 10:05:02 PM »

Any thoughts on converting mma to python 3.x? And, no, it's not just a matter of running 2to3!

I have started the project and it's not going to be trivial. A lot of things (like converting print to print() ) are trivial. And converting print output to display properly is just a PIA.

A big chuck of the time will be in converting the way internal representations of MIDI data are stored and manipulated. Since python 2.x did not have a true byte array the individual datums in MIDI data were all stored as text characters. So, to generate a 2 "character" integer we did stuff like:

     v = chr(int(x) >> 8 & 0xff) + chr(int(x) & 0xff)

which did 2 expensive (and, frankly, unintuitive) things:

     1. String concatenation ... lots of cpu time
     2. conversion to chr() ...

In python3 there is a concept of bytes. So, the above becomes:

     v = bytes([int(x) >> 8 & 0xff, int(x) & 0xff])

which generates the same 2 character array. But, it's now in bytes, not a string.

I think that just this should makes things faster. And, even if it doesn't, it'll be a lot clearer!

So, my plan at this time is to slowly convert the code to 3. Won't be done tomorrow!

But, at the same time, I'd like to modify some things in the current program to make it all nice and consistent. One example, the TEMPO command permits things like:

      Tempo +20 5.5

which increases the tempo by 20 bpm over 5.5 bars. I'd think that something like:

      Tempo +20 bars=5.5

might be more readable and in tune with other options in the program.

I have a feeling that 13.12 (the current version) will be left for use by folks with old python.

What do you guys think about all this?

     
Logged

My online life: http://www.mellowood.ca
folderol
Kara-Moon Master
****
Posts: 5296

Who? Me?


WWW
« Reply #1 on: December 31, 2013, 10:50:42 PM »

Surely the answer would be to use lists. That would then work for both. I believe each MIDI command has a maximum of 3 bytes so your basic list would be:
command = [0,0,0]
You would then make a list of lists for however many commands you want to keep current.

c_list = []
for i in range(number):
  c_list.extend([command])

you now have a 2 dimensional addressable array in the form:

value = c_list[n1][n2]
c_list[n1][n2] = value


OK, these will actually be integers, but I've used something like this for byte values with RS232 serial stuff without any problems.


you can also do whole rows:

c_list[n1] = [1,2,3]
v1,v2,v3 = c_list[n1]

... or am I missing something Roll Eyes
« Last Edit: December 31, 2013, 11:47:48 PM by folderol » Logged

If you have a poem, I have a tune, and we exchange these, we can both have a poem, a tune, and a song.
- Will
bvdp
Kara-Moon Master
****
Posts: 1436


WWW
« Reply #2 on: January 01, 2014, 12:49:52 AM »

Surely the answer would be to use lists. That would then work for both. I believe each MIDI command has a maximum of 3 bytes so your basic list would be:
command = [0,0,0]
You would then make a list of lists for however many commands you want to keep current.

Well, yes ... and well, no Smiley

midi commands can be just about any length. A byte in the preamble signals the length.

I use a lot of lists and dicts in mma. The internal midi is actually a dict of lists.

The problems arise in the funny way values are represented in midi. Long story, but has to do with the good old days when we didn't have buckets of memory. Short story, they are a complete PIA.

So, at some point one has to convert values into 2 and 3 byte STRINGS which are then included in the midi.

Having a single version of mma for python 2 and 3 was considered. I just don't think it's workable. But, we'll have to see.

When I started writing mma 9 years ago python didn't have a real byte type. So, it was all faked in strings. And, a complete pain that has been. Now, with 3 having real byte (arrays) and a lot of other nicites it'd be a real shame not to use them Smiley

So far, it looks like my 3 version MIGHT be back-portable to 2. Don't really know at this point.

Seriously, would python3 only be a game stopper for anyone? I think all the linux distros are shipping with both, some with only 3. Windows needs to dl anyway. I do think that, eventually, 3 will the standard. Just don't know when eventually is Smiley
Logged

My online life: http://www.mellowood.ca
bvdp
Kara-Moon Master
****
Posts: 1436


WWW
« Reply #3 on: January 01, 2014, 01:33:04 AM »

Damn, I get all excited about python 3 and then I read this ...

  http://alexgaynor.net/2013/dec/30/about-python-3/

Scary thing is, I think he's probably right.
Logged

My online life: http://www.mellowood.ca
folderol
Kara-Moon Master
****
Posts: 5296

Who? Me?


WWW
« Reply #4 on: January 02, 2014, 09:23:23 PM »

Bit of a chicken and egg situation really Sad
Logged

If you have a poem, I have a tune, and we exchange these, we can both have a poem, a tune, and a song.
- Will
bvdp
Kara-Moon Master
****
Posts: 1436


WWW
« Reply #5 on: January 02, 2014, 11:07:46 PM »

Damn all the chickens and eggs!

Seriously, after making this huge announcement ... I'm having 2nd thoughts. What I think will happen is that I'll work on isolating stuff which will effect the 2 to 3 change into functions which might just work with some conditionals. Print() is a biggie, but there is no problem to have all the prints in the existing program change to something like my_print(). My string/midi packing stuff is being converted right now to using the struct.pack() command in python.

So, the next version of MMA will most probably be python 2. Depending on how hard the changes are, it might be 2/3. All I can say is stand by!
Logged

My online life: http://www.mellowood.ca
folderol
Kara-Moon Master
****
Posts: 5296

Who? Me?


WWW
« Reply #6 on: January 03, 2014, 08:23:39 PM »

I agree print is a bit of a boogar. I think it was V2.6 onward that accepted both forms, but have no idea how much usage there is of older versions.

One possible solution is to use sys.stdout.write({string})

It's a bit more work, but encapsulated in your own function(s) to make sure you're converting all numbers to strings, it should work equally across all versions.
Logged

If you have a poem, I have a tune, and we exchange these, we can both have a poem, a tune, and a song.
- Will
bvdp
Kara-Moon Master
****
Posts: 1436


WWW
« Reply #7 on: February 03, 2014, 02:19:20 AM »

Just to keep the curious up-to-date, I've been working on making mma work on python 2 and 3.

At this point I have restructured the code which creates the midi data strings from using chr() to a real byte type. This should work on both 2 & 3. I have to check to make sure that no other manipulators are changing things, but I think it's pretty solid.

I have changed all the print statements to print(). For 3 this will be a function; for 2 this simply prints the contents of a solitary tuple. There will be some minor differences in the way LFs are handled, but I don't think anything important will be lost.

The import functions have been changed to work with 2 and 3.

Classes containers have been modifed.

So, we're making progress.

Next up are checking integer division. A biggie Smiley In 2 the result of 1/2 is 0; in 3 it is .5. If one wants 0 you have to use //.  This works in 2 and 3 so not a big deal.

After all that, there will be some need for conditional code (or the use of an exception) to mangle things into both pythons. But, I'm hoping that this will be painless.

BTW, with the stuff I've done so far I've found a number of minor bugs (fixed!) and managed to make the whole mess run about 1% faster. I'll be putting up a developer version in the next week or so. Hopefully you guys can find some more bugs for me Smiley
Logged

My online life: http://www.mellowood.ca
folderol
Kara-Moon Master
****
Posts: 5296

Who? Me?


WWW
« Reply #8 on: February 04, 2014, 03:14:38 PM »

Glad to see you found a silver{ish} lining Wink

I'd quite forgotten about the oddities with division. The one that caught me out for a while was getting division inside functions to produce floating point values when the function's input values happened to be integers.
Logged

If you have a poem, I have a tune, and we exchange these, we can both have a poem, a tune, and a song.
- Will
bvdp
Kara-Moon Master
****
Posts: 1436


WWW
« Reply #9 on: February 04, 2014, 05:05:52 PM »

I tried to run MMA from python3 yesterday. Still lots of things to work on. Biggest problem, for me, is the difference between byte and old-style character strings. I'm using struct.pack() to create the MIDI data and there are some oddities to overcome (that is a nice way of saying it).

Oh well, no rush. Neither python2 or python3 are going away anytime soon Smiley
Logged

My online life: http://www.mellowood.ca
bvdp
Kara-Moon Master
****
Posts: 1436


WWW
« Reply #10 on: February 14, 2014, 09:45:25 PM »

Well, congratulations to me 

I just complied a successful midi file using the python2/3 version of MMA.

Yup, mma is working with python3 Cheesy

Now, I don't know yet if any errors have crept in, and I'm sure they have. But, the midi file is playable and sounds the same as using python2.

This is huge progress and I'm pretty excited (I'm not so sure I really appreciate all the constraints a typeless language like python is putting me though to force typeness ... had I know this I might have just used C in the first place).
Logged

My online life: http://www.mellowood.ca
folderol
Kara-Moon Master
****
Posts: 5296

Who? Me?


WWW
« Reply #11 on: February 15, 2014, 08:40:25 PM »

  Grin
Logged

If you have a poem, I have a tune, and we exchange these, we can both have a poem, a tune, and a song.
- Will
bvdp
Kara-Moon Master
****
Posts: 1436


WWW
« Reply #12 on: February 15, 2014, 10:24:20 PM »

Don't get the party hats and noise makers out just yet. The python3 version only works on about 80% of the song files I have. The rest dump core ... not nice. But, the errors appear to be mostly integer division stuff. Be awhile before a developer version goes up here.

But, progress is progress!
Logged

My online life: http://www.mellowood.ca
bvdp
Kara-Moon Master
****
Posts: 1436


WWW
« Reply #13 on: February 18, 2014, 04:29:56 PM »

I have now successfully run MMA under Python 3.3 on all 1000+ files on my local computer. This includes all the example files in the distro. I've not listened to each and every file since this would be about 50 hours worth of listening ... and I do have snow to shovel!

So, it appears to be working Smiley

Only downside is that python 3 seems to run 10 to 20% slower than python 2. Not a big deal, but interesting.

I need to fix a few tests in the main mma.py program which tests python versions and some other little stuff. I'll put up a developer test version later this week.
Logged

My online life: http://www.mellowood.ca
bvdp
Kara-Moon Master
****
Posts: 1436


WWW
« Reply #14 on: February 23, 2014, 07:00:11 PM »

I have just uploaded mma-13a, a test version which supports Python 2.5+ and 3.x.

The calling script, mma.py, has a first line:

     #!/usr/bin/env python

To force the use of python 3.x you can do several things:

   1. Call the program using the command line "python3 mma.py"
   2. Change the first line from "python" to "python3"
   3. Check the file "python" on your computer. In most cases it is a sym-link to
       either /usr/bin/python2 or /usr/bin/python3. Modify as necessary

This version of mma has a few conditional switches to use 3 or 2 code (and a few is actually 2).

Would you guys please bang on this and see what I've missed.

   http://mellowood.ca/mma/downloads.html#developer

Thanks!
Logged

My online life: http://www.mellowood.ca
Pages: [1] 2
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.21 | SMF © 2015, Simple Machines Valid XHTML 1.0! Valid CSS!
Page created in 0.067 seconds with 20 queries.