Short version:
You can do it. Nothing will break.
Long version:
In the early days of transistors, input circuits were quite delicate and easily damaged by overloads. These days they are pretty much bomb-proof. The sound might not be what you want but you won't do damage.
In the main, only transducers (guitar pickups, microphones etc.) need to be properly matched impedance-wise, and even there, active, (i.e. amplified or processed) versions don't need matching. Indeed, for most interconnection work 1:10 is the usual rule of thumb for distortion reasons. So, an effects unit with an output impedance of, say 5k Ohm wants to feed into an input of 50k Ohm or higher.
The exception is balanced 600 Ohm studio stuff where to source is exactly 600 Ohms and need to be fed into exactly 600 Ohms for the right matching of signal levels, line characteristics (if the cables are long) and interference suppression.
More problematic than impedance is signal level. There seems to be no real standard about these where effects units synths etc. are concerned - I'm getting a bit out of date here so things may have improved - yeah right
As well as overload problems, if there is a large signal mis-match noise levels will climb, and if you have to put attenuators in, then you are doing unnecessary amplification somewhere, which is going to be bad for both noise and distortion.
<edit>
Re-reading that I'm beginning to sound like one of my old lecturers . Eeeek
