>>>> "Hugo" == Hugo Andrade Cartaxeiro <hac@nexus.inesc.pt> writes: : :Hugo> print $str; :Hugo> eir 11 9 2 6 3 1 1 81% 63% 13 :Hugo> oos 10 6 4 3 3 0 4 60% 70% 25 :Hugo> hrh 10 6 4 5 1 2 2 60% 70% 15 :Hugo> spp 10 6 4 3 3 1 3 60% 60% 14 : :Hugo> and I like to sort it with the last field as the order key. I know : :Hugo> perl has some features to do it, but I can't make 'em work properly. : :Well, speaking Perl with a Lisp.... :-)
We all recall that Perl est un esprit LISP dans un corps C, n'est-ce pas? :-) (``corps C'' sounds like ``corset'' in French.) [Philippe Verdret <phd@eurolang.fr>]
Oh for cryin' out loud, Randal! You expect a new perl programmer to make heads or tails of that>? :-) You're postings JAPHs for solutions, which isn't going to help a lot. You'll probably manage to scare these poor people away from the language forever? :-)
BTW, you have a bug.
Couldn't you at least indent it properly? :-) This might help folks make more sense of it. I'm going to use the
rather and then
of map here so that it looks more like a conventional subroutine call (although I can't do that with sort):
Hm... ok, not much better. Let's do this slowly and work our way through it. First of all, let's look at your string again.
It looks like it's got a bunch of lines in it, so the first thing we're going to do is break it up into a list of lines.
You need to do this because each the perl built-in sort() function expects a list and returns a list. It works like this in its most simple incarnation.
Well, that's all well and good, but it's going to do the default sort, meaning it'll sort it by ASCII. Imagine a function like this:
You could then say
It turns out that the $a and $b are each of the two values to be compared in the original list. They're not going to passed in the way normal arguments are, which is $_[0], $_[1], etc.
That's work out ok, but it's pretty complicated looking. It turns
out that for just a matter, perl has a couple of operators to help
you: <=> for numbers and cmp for strings.
While normally we give split a regular expression as its first argument,
there's one case where you might want to give it a string: if the first
argument is just a single space, then it will automatically discard
leading null fields, which would otherwise be preserved. (Trailing null
fields are discarded in any case (unless you give split() a third
argument).)
Now you could also have written that function ``in-lined'', like so.
I'm going to leave out the word return this time:
Ok so far? We can make use of perl5's ability to do negative subscripting to get the last element:
And in fact, we can do this without the temporary lists:
But no matter how you write it, that's a wee tad expensive for large data sets. Since Perl's sort is a hook into C's qsort (lookup qsort(3) in the on-line Unix manual, if you're daring), that makes it N-logN sort. You're going to be doing 2 splits for each of those. With your data set of four, this isn't two bad: you'll average only 11 splits. With ten lines though, you're already up to 46. With a mere one hundred lines, 921 splits, and with a thousand lines, we're talking 13,816 splits on the average! That's way too many.
So let's arrange to do just N splits. We'll have to pre-process them:
Now for a given line $n in @oldlines , its last field can be found in the corresponding element of the @last_field array, that is, $last_field[$n] .
You could also just use the return value from split directly, without using a temporary, like this:
You'll find that many perl programs are used to letting the $_
variable be the default to quite a few different operations. This is
unsettling to the newcomer, but it's really not that bad. Amongst other
things, $_ is the default iterator variable on a foreach loop, the default
string to split on in a split() function, as well as the default variable
to search or replace in a match or substitution. (You'll find that it's
also the implicit iterator variables in the
map() and
grep() built-ins, which are themselves something like short-cut ways
of writing
foreach loops.)
In perl5, we can write that without so many parentheses if we'd like. This time I'm also going to spell "foreach" as "for". It actually doesn't matter to perl, but it might to the reader.
This entire process of reducing things to smaller and smaller amounts of code to do the equivalent thing is an interesting exercise, but perilous. While between 1.1 and 1.5 and between 2.1 and 2.4, we actually did improve the efficiency of the operation ever so slightly, we've also managed to make it harder to debug and harder to maintain. It's nice to have intermediary values sometimes in order to print them out part way through the calculation, whether in the debugger or with normal print statements. It's also harder for someone to come by later and figure out what you're doing. If your aim is job security, then this might work out in your favor, but in most other cases, it probably won't.
Ok, how do we couple the stuff we did in code set 2 with what we did in code set 1? Remembering that any given element in @last_field corresponds to the last field (the thing we want to sort on) in the @oldlines list. So we're going to do something a bit strange and sort not real data but rather just the indices. That is, we'll rearrange indices (like 0,1,2,3,4) to correspond to the properly sorted order (like 3,2,1,4,0). This is probably the hardest concept to grasp in this whole tutorial.
Now when we're done, we've got everything into its proper order. Occasionally, advanced perl programmers make use of what we call array slices. If you say
CODE: 3.2.1:
It's the same as though you'd said:
Which is really nothing more than simply:
Except that it's shorter to write. Notice how when you're doing slice operations you have to subscript your list with an @ not a $? That's because the $ or @ sign actually indicates what it is that you're getting back, not what you're looking at. In this case, you're assigning a (sub-)list to a (sub-)list, so you need the @ signs.
This means we can re-write 3.1 as
Now, since sort will return the indices, we can avoid some temporaries and write that as:
If we'd used shorter variable names, we could have then written it all on one line:
Now what? Well, we have to get back the user's orginal data.
But you'll find that you're shy a final newline (that's Randal's bug).
You could solve this by passing another null argument to join() ,
or by concatenating a newline to the result:
Combining all these, we have final solution that looks like this:
We can merge a few of these and have something shorter.
It's true tha this is verging on the inscrutable, but we're trying to get closer to Randal's obfuscated perl entry he presented way at the beginning of this.
Nonetheless, it's still quite a bit different from Randal's. One thing he's done is make use of the map() function. This is a perl5 feature that uses its first expression argument as code to execute repeatedly against each of its following list arguments, setting $_ to each element, and returning the result of the expression to create a new output list.
You may also write this expression as a block, and omit the comma, which is what Randal did:
Or even omit the temporary:
You can use the map() function to shorten up 5.2 a bit by pulling
out the for loop:
You could even make use of the result of the assignment statement as the argument to map and shrink it sill more:
Now, Randal didn't want to use any temporary variables at all. Let's look at his jumbled code again:
He's been kinda nasty in that there are no parens or indentation. I offered this version which adds them:
But you could retain Randal's block form as well:
While in some ways code 0.2 resembles code 5.4, in many others it doesn't. What are all those crazy arrows and square brackets doing there? Well, that's a really interesting question.
Let's take this thing inside out. Here's the innermost portion of it:
What's going on is that he's using the return from split to be the arguments to map() over. Ok, but what's that map() code block doing???
What 6.2 does happens to do is to create a reference to a list of two elements. It's as though you said this
However, the list reference he's creating by using the [ .... ] notation isn't a named list at all. If you wrote the inner loop as a real honest-to-goodness loop, it might look like this: (remember that 6.3 is just the long form of 6.1)
And then you'd have all your references in @listrefs. This means that for a given line within $str (call it line number $n ), $listrefs [$n]->[0] is going to be the whole line itself (without its newline), while $listrefs[$n]->[1] is going to be its last field. We could actually write those without the pointer arrows, because adjacent brackets may omit them, as in $listrefs[$n][0] , but we're going to need them for what later, so you should get used to seeing them.
Well, what's the next level do? It's a sort. Consider this:
Since $a and $b will be each element of @listrefs, they will each be references to lists (kinda like pointers to lists). Remember that each of these refers to a list of two elements, whose zeroth element $a->[0] is the entire data, but whose first element $a->[1] is the sort key. So we'll compare the sort keys.
Put we don't want the sort keys. We want the original data. You could write a loop to do it:
Or you could be like Randal and avoid temporaries by using a map() instead of a loop:
Combining up various stages, we get this:
It may not look like it, but this is very, very close to what Randal wrote back in 0.0; he just didn't keep around his temporary results in names variables. It's quite likely that he even developed the 0.0 code starting from something that looked more like 7.4. If you don't, you'll have an exceedingly difficult time debugging it.
But you know something? You don't have to be all so super-clever to do this. There's a much more straightforward solution:
See? We just write it to a file, then let the system sort program do the work for you. Isn't that much easier to understand? If you can easily express your sort in something that the system sort program will understand, it usually makes more sense to use it. It'll run faster and be easier to write as well.
Of course, we've made committed some minor errors here by omitting error checking. This would be better:
That line to print() looks funny, eh? And what about the parens on the unlink? You probably thought that you could have written both of those as
But in fact, you couldn't. The || binds more tightly than you want. It would have been as though you'd written:
Which certainly isn't good. That's why in perl5 we have or, which is
a very low-precedence operator. We probably would write it like this:
Another trick sometimes used is to use to the filename itself as an indirect file handle.
Another trick sometimes used is to use to the filename itself as an
indirect file handle. This has a greater advantage when you are reading
from the file, and issuing diagnostics using warn()
or
die(), which include the line (or chunk) position and filehandle anme
of the last file read from. This way you'd also get the
Now the warn will include the actual filename rather than just the filehandle, like TMP.
Now, you probably shouldn't bother go to too much trouble in figuring out a good filename. There is a POSIX function which will do this for you.
Just one final thing. This seems like just the application for a
bi-directional open. You might be tempted to write this using the
open2()
function.
or maybe to read the whole thing back again, undef the RS:
Now, it turns out that in this case it actually works, at least on my system. I wrote all my input do the pipe first, which gladly read it all in up until end of file, and then spit it out. That's how sort work. But cat isn't so friendly, and works far more like most programs.
If you do this and $str is big enough, you'll block forever. And even doing a piece at a time:
use IPC::Open2;
$kidpid = open2('INPUT', 'OUTPUT', 'cat');
while (1) {
print OUTPUT $str;
$newstr =<INPUT>;
print "just read <<< $newstr >>>\n";
}
This approach won't work because in general, you can't control your child processes notions of buffering, which are entirely useless for simulating interactive work as you're attempting here. The only programs this works on are ones that are designed for it, like dc. There aren't many system programs like it.
I strongly advise against using open2 for almost anything, even though I'm its author. UNIX buffering will just drive you up the wall. You'll end up quite disappointed. It's better to use temp files as we did back in the code examples in section 9 if you really want to deal with an external process in which you wish to control both its input and its output.
There's one final lesson I'd like to impart. It's not a bad idea to order your coding priorities like so:
Notice at no point did ``4. cleverness'' enter the picture. Of course, if you're writing Obfuscated C contest entries, or tricksy little JAPHs, then it makes perfect sense to invert these priorities, like so:
But that's only good for tricks, not solid code, which is why code like that in 5.1, 8.3, or even 7.4 will in the long run make easier both your life and the lives of those around you.
While a LISP or Scheme programmer may be entirely conformatable with arrange their code as Randal did in 0.0, I haven't personal found that most of your average off-the-street UNIX, DOS, VMS, or IBM programmers take very quickly to that style of functional programming -- or even the above average ones, for that matter. It's nice in its own pristine way not to use any temporary variables and all, but it's still not to most people's liking.
And no, I'm not ragging on Randal, merely teasing a bit. He's just trying to be clever, and that's what he does. I'm just submitting a sample chapter for his perusal for inclusion the mythical Alpaca Book :-)
--tom