Kara-Moon Forum

Developers & Technology => Musical MIDI Accompaniment (MMA) => Topic started by: sciurius on December 31, 2018, 04:05:45 PM



Title: Outsmarted
Post by: sciurius on December 31, 2018, 04:05:45 PM
I have a plugin Massage that currently just copies the arguments to a new input line.

    @Massage 1  Cm7

works as expected. But then:

    Begin @Massage
        1  Cm7
        2  Dm7
    End

Well, I guess you can imagine what happens... Actually, it is quite nice it works this way, but in this particular case it gets in the way  :D.

I think being able to apply a plugin to a series of lines using Begin/End is powerful, but how to get it to work  ???


Title: Re: Outsmarted
Post by: bvdp on December 31, 2018, 04:32:51 PM
Oh, don't like recursion?

Funny that you mention this ... I got stung the other night by the same issue ... and the docs now reflect this (as of last night) ... I'm a step ahead of you this time :)

I think you have to apply the function for each line. I don't see a way just now to avoid this without a major rewrite to the code.


Title: Re: Outsmarted
Post by: sciurius on January 04, 2019, 09:35:52 AM
Actually there is a very simple, even elegant solution.

Currently, plugins do not return a value. What if they did?
What if they could return a single input line?

Then the current code:

Code:
        if action in simpleFuncs:
            simpleFuncs[action](l[1:])
            continue

could be modified to:

Code:
        if action in simpleFuncs:
            l = simpleFuncs[action](l[1:])
            if l is None:
                continue

Similar for the trackFuncs and dataFuncs entry points. In other words, the plugin would become a real line 'filter'.

It is not as dangerous as it looks, since at the point where the plugins are invoked the input line is only split and no real work has been done on it.

The only (and sensible) restrictions would be that a data plugin should return a line without label, and a track plugin should return a line without track.

What do you think?


Title: Re: Outsmarted
Post by: bvdp on January 04, 2019, 04:21:46 PM
I think there would need to be a restriction that only single line returns are permitted ... the way it is used just now one can shove multiple lines onto the input stack and the parser will process them in turn.

Certainly something to think about.


Title: Re: Outsmarted
Post by: sciurius on January 04, 2019, 09:30:42 PM
Yes, that would be a difference between returning a line and pushing lines back into the input. But one does not exclude the other, you can still push lines but these are then subject to Begin/End prefixing and will cause unlimited recursion.


Title: Re: Outsmarted
Post by: bvdp on January 04, 2019, 10:24:49 PM
It would sidestep the issue with infinite loops in Begin/End. Nice idea.

Consider it done. And, we'll leave the "don't do multiple lines" issue up to the writer of the plugin :)

I think we can apply this to all 3 types of returns. Let me play with it over the weekend (between my home painting projects).


Title: Re: Outsmarted
Post by: bvdp on January 04, 2019, 11:30:18 PM
Just looking a bit more at the code ... and I don't think it's quite as easy as first blush :)

First off,

        if action in simpleFuncs:
            l = simpleFuncs[action](l[1:])
            if l is None:
                continue

Our new 'l' will now drop down to be tested by trackfuncs and data. It will not be re-interpreted as a simplefunc ... so that gets confusing.

Next, for track and data the same will apply.

It might work if we wrap "if action ... " into a loop which continues until l is none?

Oh, you guys, you make it sound so easy :)


Title: Re: Outsmarted
Post by: sciurius on January 05, 2019, 07:41:08 PM
It will not be re-interpreted as a simplefunc

I considered that to be a feature. The processing of input lines has a number of steps. First is macro expansion, second is begin/end prefix adding, then simple function testing, and so on.

When a plugin does not return a value, processing ends there and the main loop continues with the next line of input. If the plugin returns a value, processing continues with the next step.

Quote
It might work if we wrap "if action ... " into a loop which continues until l is none?

I invented this tweak mainly to bypass the begin/end prefix adding so "begin @plugin" could be made to work. We can discard the whole return value trickery when there is a good alternative way to do this.

For example, the plugin could set a (global) flag that the next N lines must not be prefixed. But this could result in even more trickery...

I'll give it some rethinking...


Title: Re: Outsmarted
Post by: bvdp on January 05, 2019, 09:52:27 PM
The question I have to ask is "Why are we doing this?" Is it all just to avoid recursion when the code is executed inside a begin/end block? And if that is the concern, it would be trivial for the plug code to strip whatever has been added by Begin from the current line and send that back and then let begin add it back again. Would that work?

I have played around with setting flags around the code adding the begin stuff ... and it really doesn't work very well.



Title: Re: Outsmarted
Post by: bvdp on January 05, 2019, 10:25:15 PM
Please ignore my last post. It's stupid.

If we modify the simplyfunc caller to this, it might work:

Code:
while  action in simpleFuncs:
         l = simpleFuncs[action](l[1:])
         if l:
            action = l[0]
         else:
            action = None
 

The idea, I think, is to loop over the simplefunc until it returns None.

Now, for trackplugs we could do something similar. The restriction would be that the TRACK could not change.

Dataplugs are fine as is.

Thoughts?



Title: Re: Outsmarted
Post by: sciurius on January 05, 2019, 11:00:47 PM
I come to the conclusion that the current plugin model is flawed anyway.

Consider the following:

Begin Chord
    Command1
    Command2
End


is lexically identical to

Chord Command1       // track command Command1 on track Chord
Chord Command2       // track command Command2 on track Chord


Now consider a track plugin Plugh that pushes back a track command.

Chord @Plugh             // track plugin @Plugh
Chord Command            // line pushed back by @Plugh


So far, so good.

But now:

Begin Chord
    @Plugh
End


will be interpreted as

Chord @Plugh              // @Plugh + prefix
Chord Chord Command       // line pushed back by @Plugh + prefix


So if the plugin prefixes the track to the pushed back comand, which I think is the right thing to do, then it will fail in the Begin/End case.

A clean solution I see is that lines pushed back by a plugin must not be prefixed with the begin/end prefix.

This solution eliminates the need for 'plugins return a value' and also solves the unwanted recursion in the "Begin @Plugh" case.

The implementation is a bit more complex, since function parse (parse.py) must somehow find out whether the input line came from a push back. But this is certainly doable.


Title: Re: Outsmarted
Post by: bvdp on January 05, 2019, 11:57:43 PM
There are a number of cases where mma uses the pushback stack, not just for plugins.

So, the parser would need to know where the next line is coming from ... is it a plugin pushback, other pushback or new line. And, now we are hopelessly complicated and error prone.

I spent some time with the loop code I posted earlier. It works, but I've not tried it for track commands. But, the problem is that the cases for using are getting so few and far between I'm having trouble seeing it's usefulness. The way the plugins work right now is pretty solid ... just don't do it in a begin/end block. And, yes, it is possible for the plug code to check to see if this is the case and to issue a warning. Heck, we could even have parse do a check and issue a warning if the command starts with "@".

What kind of real code are we trying to write with these mythical plugins? Perhaps they are better done with macros or subroutines?




Title: Re: Outsmarted
Post by: sciurius on January 06, 2019, 12:39:55 PM
I would advise to use a distinct pushback list for the plugins.

Pushing lines back to the input is okay as long as it does not make a difference where the next line originated from.

As I tried to point out, bypassing begin/end prefixing is only relevant for pushback from plugins. Since plugins are encouraged to use the API provided through pluginUtils.py it is trivial to change its sendCommands method to push the lines to the plugins pushback list, and document that pushing back lines to the input may yield surprising results.

Quote
The way the plugins work right now is pretty solid ... just don't do it in a begin/end block.

The suggested changes extend the solid use of plugings with begin/end blocks.

Quote
What kind of real code are we trying to write with these mythical plugins?

The thing that triggered all this is the revoice plugin I wrote. It takes an ordinary input line and replaces the chords by random voicings. For example:

  @revoice 24 Cm7 Fm7 Cm7

would push back:

  DefChord m7-0 (0, 3, 7, 10, 12, 19) (0, 2, 3, 5, 7, 8, 10)
  DefChord m7-1 (0, 3, 7, 10, 15, 19) (0, 2, 3, 5, 7, 8, 10)
  DefChord m7-2 (0, 3, 7, 10, 12, 15) (0, 2, 3, 5, 7, 8, 10)
  24 Cm7-0 Fm7-1 Cm7-2


While @revoice 24 Cm7 Fm7 Cm7 works as always, it just seems more elegant and intuitive to write:

  24 @revoice Cm7 Fm7 Cm7

And, in particular:

  Begin @revoice
     24  Cm7 Fm7 Cm7
     25  Cm7 Cm7 Gm7
  End




Title: Re: Outsmarted
Post by: bvdp on January 06, 2019, 10:31:12 PM
Did you hear the one about the fellow who went to the doctor and complained that whenever he moved his had a certain way it would hurt ... the doctor told him "don't move that way" :) I think the issue with pushbacks is entering a similar area ...

Here's the problem: Yes, we could set things up so that when you have this:

Begin @myplug
  10  Dm Em
  20  A  B
End

the plugin avoids the pushback problem. It's not all that easy: not only do we need a different pushback stack, but we need for the input loop to check in 2 locations as well, and skip Begin prefacing when it comes from one spot and not the other.

Now, what happens when:

Begin Chord
   Begin @myplug
      ....
    End
End

I assume we only want to bypass prefacing @myplug, but continue to preface "chord". I'm getting confused already.

Would having the begin-preface code NOT append plugin names do anything?

I'm going to leave this for a few days and go work on random voicings :)



Title: Re: Outsmarted
Post by: sciurius on January 07, 2019, 07:36:49 AM
I do not see the problem.

 Begin Chord
    Begin @plugh
       10 Cm
    End
  End


is equivalent to

 Chord @plugh 10 Cm

So it calls the trackRun entry point, and the plugin pushes back lines prefixed by the track.

The main loop should therefore prefix nothing to pushed back lines, since it has all be handled.

But maybe I'm overlooking something crucial?



Title: Re: Outsmarted
Post by: bvdp on January 07, 2019, 04:14:43 PM
Like I said, I'm going to let this gel for a few days. I'm sure I'm seeing problems which aren't there :)