...making Linux just a little more fun!

<-- prev | next -->

Mailbag


Mailbag

Adding a Broadcom NetXtreme Gigabit Ethernet Driver
soundcard
regarding simulation of ARP
Problem in Linux FC5 Kernel recompiling
Blue line
Colouring $PS1...
Adding bogofilter to Exim4: ""doesn't work"".
Starting to explore the Linux kernel code
Message sending failure

Adding a Broadcom NetXtreme Gigabit Ethernet Driver

DR James (vas_an_nox at yahoo.com)
Tue May 30 07:31:54 PDT 2006

Answered by: Thomas

Greetings James,

How do you load a driver for an integrated network interface card in RedHat Enterprise? The driver is for a Broadcom NetXtreme Gigabit Ethernet Driver.

Thank you!

[Thomas] - It's been a while, clearly. We're a "gang" now.

You'd use "modprobe" like everyone else. Ascertaining the name of the module to load is left as an exercise to yourself. :P

[[DR James]] - Thanks Thomas!


soundcard

(Jazzyjayne69 at aol.com) Jazzyjayne69 at aol.com
Sat Jun 10 22:16:10 PDT 2006

Answered by: Ben, Marty, Rick, Thomas

I need a soundcard for win 98 download free if possible...PLEASE help me,

Jayne

[Thomas] - No. At least, you're out of luck here. You randomly emailed a group of us that answer questions about Linux. Note that I assume you mean a "driver" for A soundcard, since one doesn't download soundcards directly on the Internet. God forbid you should ask that exact question anywhere else, but even the name of the soundcard would help.

Good luck. Your answer is anywhere but here.

[[Marty]] - Maybe he wants the schematics? ;-)

[Ben] -

--------------------------------------------------------------------------
* * * * * * * THESE ARE NOT THE DROIDS YOU'RE LOOKING FOR * * * * * * *

The address to which you have sent your query is that of The Answer Gang 
for the Linux Gazette. We answer questions related to the Linux operating
system. Unrelated questions are likely to get anything ranging from a 
polite answer to astonished laughter, with curmudgeonly replies being 
somewhat of a norm. Good karma points are awarded to those who tell us 
where they got our address and the reason they thought that we might help 
with fish identification, spaceship repair, or nude snowboarding. 

Spammers will be whacked with a dead trout; flames will be graded for 
content, laughed at, and sent to /dev/null; amusing bits may get shared 
with our readers.

Good luck with your search.
--------------------------------------------------------------------------

[Rick] - Volley:

I need a source for good pastrami on the USA West Coast.


regarding simulation of ARP

senthil kumar (senthil_sma at yahoo.co.in)
Wed Jun 28 02:36:53 PDT 2006

Answered by: Ben

Respected sir,

I am senthilkumar from India,

My question or request is please send me a Cprogram for simulation of ARP and RARP

thanking you

[Ben] - Sure, umm... senthilkumar from India. However, we first require your professor's name and email address; when we do someone's homework for them, we want the credit - and the degree or the certificate for the course as well. If you wanted those, you'd be doing the work yourself; since you're so kind as to offer them to us instead, we can do nothing less than accept gratefully.

Please send the information as soon as possible - I just know everyone's fingers here are itching to start typing '#include <stdio.h>' and 'int main(int argc, char *argv[])' right away!


Problem in Linux FC5 Kernel recompiling

Deepak Mishra (mishradk at gmail.com)
Mon May 29 04:58:06 PDT 2006

Answered by: Deepak, John, Thomas

Hello,

I had a FC5 installation on my PC. now to enable RTAI i am going to recompile the source code with the patch.

Now my question is when i do a "make modules_install install" after compiling the kernel source, is it going to replace my old .ko files? or my old modules will also be there?

If it is not going to replace my old .ko files then where it is going to install the new modules and how it knows which module is for which version? ( i have disabled version support)

and if it is going to replace my existing .ko files the how my old kernel will start if i select the old kernel from grub menu while booting?

i have searched the Internet but did not get proper answer.

Hope to get some answer for this. Regards Deepak

[Thomas] - [Quoting Deepak Mishra]:

  > Hello,  
  > I had a FC5 installation on my PC. now to enable RTAI i am going to  
  > recompile the source code with the patch. Now my question is when i do  
  > a "make modules_install install" after compiling the kernel source, is  
  > it going to replace my old .ko files? or my old modules will also be  
  > there?

Assuming you have a copy of this kernel already installed, the files will just get installed over the top of it.

  > If it is not going to replace my old .ko files then where it is going  
  > to install the new modules and how it knows which module is for which  
  > version? ( i have disabled version support)

But by disabling "version support" that doesn't preclude the fact that files are going to be installed to /lib/modules/$(uname -r).

  > and if it is going to replace my existing .ko files the how my old  
  > kernel will start if i select the old kernel from grub menu while  
  > booting?

Depends how you configure grub.

I am not sure of what answer you're after, or indeed, why it matters.

[[Deepak]] - Hi Thomas,
Thank you very much for answering my problem.. Actually i want to ask one thing is that, I have already compiled a stable kernel. It is working fine.

Now i have made changes in to some important parts. In my previous kernel also I had compiled this code as a module and now after making some modifications I am recompiling the kernel with a different version.

I think after installing the new compiled kernel and its modules, it will over write my my previous module.

[[[Thomas]]] - Right. Well, the heuristic you're using is that if you're changing some part of your kernel configuration which is for the same version as the kernel you're currently using that you shift

 /lib/modules/$(uname -r)

out of the way before you issue a "make install".

[[Deepak]] - After booting from my new kernel I found that my new module is not working as per my requirement. At this point if I boot my system with the previous kernel, can I go back to my old working stable setup? or I have to recompile my code after removing my code?

[[[Thomas]]] - Assuming you do the above, it's generally safer to issue a "depmod -a" command. This typically happens during startup. Note that if what you're compiling is just modules for the current running kernel, you theoretically only need install those modules, as in:

 sudo make modules_install sudo depmod -a modprobe some_module_name

[[Deepak]] - I am new in this Linux kernel compilation. so please advice me.

[[[Thomas]]] - It's a topic that has come up before, and I believe the kernelnewbies.org site is quite good also.

[[[[Deepak]]]] - Thank You very much for your advice and suggestions.

[John] - I'm not 100% sure of it, but I think the following method will not overwrite your existing modules:

- In the kernel configuration menu, choose "General setup"

- Then "Local version - append to kernel release"

- Input a string (64 char length max, but you'll probably want to make it quite a bit shorter than that).

Then after booting from that kernel, "uname" command will return the string you entered in the step above appended to the usual kernel version number. The modules corresponding to that kernel should be installed under /lib/modules/KVER-localVersion where localVersion is the string you entered during kernel configuration.

I say I'm not 100% sure here because I haven't used that method since trying it once back in the days of kernel 2.0 or 2.2 at least 8 yrs ago, and my memory of it is a bit tenuous.


Blue line

Alice Perrault (aaperrault at adelphia.net)
Tue May 30 17:04:31 PDT 2006

Answered by: Ben, Brian, Dale, Thomas

Why do I sometimes get a vertical blue line on the left side of my message. Of course, it wouldn't happen this time, when I want it to show. Many thanks for any answers you can give to me. ~~ Alice

[Thomas] - In what? Email? Which client are you using? Is this blue line rendered with the text you're reading? Is it some form of colourisation based on syntax?

You need to be much more specific -- you don't give us much to go on.

[Dale] - Might be a form of quoting a previous message, as in a reply. As Mr. Adam says, the answer is in no small measure dependent upon the context of your question.

On another note, when posting to a list like this one, it is generally considered proper protocol (at least in my experience) to send the message as plain text. Many folks use text-based mail clients like Mutt or Pine and Rich Text/HTML messages can be inconvenient to deal with.

[[Thomas]] - You can call me Thomas.

What's even better "protocol" is people learning NOT to top-post.

[[[Dale]]] - Bad habit I picked up long ago.... sorry.

[Ben] - Alice, I'm going to take a wild guess and say that you're using 0utlook under Wind0ws - which uses a blue line to denote quoted text. If this is the case, I hope that the above answers your question, because we normally don't answer Windows questions: The Answer Gang is a part of the Linux Gazette, and answers questions about Linux.

If I've guessed wrong, then I invite you to read "Asking Questions of The Answer Gang" at http://linuxgazette.net/tag/ask-the-gang.html and write us back - hopefully with your question rephrased as per the guidelines (i.e., including sufficient information), and we'll see if we can help.

[[Brian]] - I get a blue line for quoted text in Mozilla Thunderbird under Linux when T'bird is being "polite" and rendering HTML email instead of doing everything plaintext as I instruct it. To be more explicit, when I'm looking at a reply to an HTML email in Mozilla Thunderbird in the message pane, the bits that are quoted from prior espondent(s) are bordered on the left with a vertical graphical line. The color of that line changes with depth of quoting...

For example, in a later message in this thread, where Dale quotes Thomas, there is a blue line down the left side. Where Dale quotes Thomas quoting Dale, there is a DarkRed line inside the blue line, indicating two levels of quoting. In a plaintext message, I would instead see two greater-than symbols (that is, '>').

It is most irksome to me that these lines don't revert to plaintext quoting methods, even when I explicitly instruct T'bird to display the messages in plaintext. The TLA "WTF" sproings to mind...


Colouring $PS1...

Mahesh Aravind (ra_mahesh at yahoo.com)
Sat May 13 03:30:05 PDT 2006

Answered by: Ben, Kapil, Martin, Thomas

Dear fellow Linuxers,

I had recently been fooling around with the colouring the bash(1) $PS1 variable for making $PS1 really colurful prompt. I found myself reading the console_codes(4) man page (and the inevitable bash(1) man page, too) from time to time.

This was a pain. As for a newbie, s/he can't quickly grasp the concept of DEC escape codes, and differentiate VT220 codes from ECMA control codes. So to help the newbies, I decided to code up a simple(?!) shell script from which the desired colour codes could be looked up as X,Y coordinates.

(Download script
from here.)

This is no great hack. And I concentrated only on the `Normal' and `Bold' category of codes, since `Underline' and `Blink' attributes couldn't always be handled by all VCs (of course, some xterm(1x) could interpret them).

In the script, the Y axis denotes the foreground colour ranges, and the X axis denotes the background colour ranges.

Well, the standard disclaimer applies. Any suggestions and corrections are welcome (all flames to /dev/null). I'm releasing this script as GPL.

No worries,

Mahesh Aravind

PS: If you want any help, do an "nl -ba colors.sh". ;-)

[Ben] - Very nicely done, Mahesh! I wrote a similar function in my Bash tutorial; I've tweaked it quite a lot since then. Yours is prettier, though, and includes the 'bold' versions.

  colsel () # Color selector - iterates through a $TERM's color choices 
  { 
  trap 'reset' 0 # Reset on exit 
  n=47 # Max foreground color value 
  while [ "$n" -gt 39 ] 
  do 
  m=37 # Max background color value 
  while [ "$m" -gt 29 ] 
  do 
  [ `expr $m % 20` -eq 0 ] && echo -e '\E[$40;1m|' 
  echo -en "\E[$m;${n}m| $n $m \E[$40;1m" 
  m=`expr $m - 1` 
  done 
  n=`expr $n - 1` 
  echo -e '\E[$40;1m|' 
  done 
  } 

Oh, and just because I can't resist:

  perl -we'for$a(40..47){print"\e[$a;${_}m$a $_|"for 30..37;print"\n"}' 

[grin]

Conversely, you could just use Moshe Jacobson's "color" program; it'll even show all the colors in much the same sort of layout when invoked with the '--list' option.

  > Well, the standard disclaimer applies. Any suggestions and corrections  
  > are welcome (all flames to /dev/null).

It may already be filled up; that's where we're sending ours...

  > PS: If you want any help, do an "nl -ba colors.sh". ;-)

I've always found 'vi' to be just as useful. :)

[[Thomas]] - I've always avoid the use of escape codes in favour of tput(1) -- at least then it works remotely, too.

[[[Ben]]] - How would you use 'tput' to set colors? I've just run 'infocmp', and it doesn't show anything related to doing so.

[[[[Thomas]]]] - tput(1) can query the terminal database, hence:

  tput setaf 1; echo this is red 
  tput setaf 2; echo this is green 
  tput setaf 0; echo now we are back in black 

etc...

[[[[[Ben]]]]] - Ok, I got you. Even though "tput" options aren't listed anywhere near the "tput" man page (for anyone interested, they're in "man 5 terminfo"), and there is a huge number of them with mostly-cryptic explanations of what they do, this is still an excellent tip.

So -

for n in `seq 0 7` 
do 
        for m in `seq 0 7` 
        do 
                tput setab $n    # Set background 
                tput setaf $m    # Set foreground 
                echo -n "$m $n" 
                tput op          # Reset to original color pair 
                echo -n "|" 
        done 
        echo 
done 

The data displayed can be used with 'tput' invocations to set the colors shown, much like Mahesh's script, but would be term independent. Pretty sweet...

[[Martin]] - Just to let you guys know that there is a BASH Prompt HOWTO at tldp.org which gives you more info on colouring your prompts and putting other things in there too... ;)

[[[Ben]]] - I recall reading it a couple of years ago. Unless it's been updated, it uses escape codes - which, as Thomas pointed out, aren't necessarily term-independent.

[[[Mahesh]]] - [Quoting Thomas]:

  > I've always avoid the use of escape codes in favour of tput(1) -- at
  > least then it works remotely, too.

Yeah, but this was inspired by reading the console_codes(4) man page and seq(1) info page. Combining the two resulted in a useful(?) script.

[[[[Thomas]]]] - There's nothing wrong with it, but hard-coding escape codes is going to turn around and bite you on the arse before long. Some terminals don't/can't render escape codes properly.

[[[[[Mahesh]]]]] - Just out of curiosity, how would a modified script using tput(1) would be able to print the colour coordinates to the o/p?

[[[[[[Ben]]]]]] - Just like using escape codes does - except you don't use the escape codes. :) The point to 'tput' is that it actually queries the terminfo database and outputs the correct escape codes for a given terminal. Using explicit escape codes will work much of the time - after all, many Un*x terminals use the same ones - but will fail miserably on terminals that use something different.

If you're asking about an example, here's one:

tput smso		# Turn on bold mode
echo This should print in bold
tput rmso		# Turn it off

On the other hand, if you use these a lot, you're going to get tired of typing - so it makes sense to request them once and save them:

a=`tput smso`
b=`tput rmso`
echo "${a}This should print in bold$b"

[[[[[[Thomas]]]]]] - The o/p being what or whom? (Abbreviations, unless they're generally well-recognised, are pretty much useless for communicative writing).

[[[[[[[Mahesh]]]]]]] - Wow... looks like you're in a hot mood! (uh, read: angry)

[[[[[[[[Thomas]]]]]]]] - Far from it. Welcome to the ambiguity of email.

[[[[[[[[Ben]]]]]]]] - Nah. The last time Thomas got angry... well, you heard of what happened at Troy, right? When he gets angry, it's obvious (although the people to whom it's obvious may not have much time to regret it.)

In other words, don't worry about it. Unless you've already been slaughtered and buried in an unmarked grave - in which case, disregard everything I've said. :)

[[[[[[[Mahesh]]]]]]] - o/p == out put

[[[[[[[[Thomas]]]]]]]] - Ah, I see. In the English language, "output" is all one word.

[[[[[[[[[Mahesh]]]]]]]]] - Here's some more that I'll be (or someone else'll be) using

	w/o == without
	h/w == hardware
	s/w == software
	s/o == Significant Other (*not* son of)
...

[[[[[[[[[[Kapil]]]]]]]]]] - Why not "son of"?

Anyway, you seem to be missing the point. It is generally up to the communicator (in this case you) to ensure that abbreviations that you use are known to the communicatees (in this case not just Thomas or TAG but possibly other readers of LG).

Otherwise it is rather like sending an ogg file to someone who uses a Macintosh or a Word file to me. The recipient might be able to figure out what you are trying to say---but might press the delete button instead.

[[[[[[[Mahesh]]]]]]] - How do you get the colour coordinates using tput?

[[[[[[[[Thomas]]]]]]]] - Coordinates for what, or where? All tput does is query the terminfo database.

[[[[[[[[[Mahesh]]]]]]]]] - Uh, ... fotget it!

[[[[[Mahesh]]]]] - And how would you create (a hacked up version) of files like, say, /etc/issue using tput w/o using hard coded escape codes?

[[[[[[Ben]]]]]] - Those are not terminal codes, they're 'getty'-specific escape codes; no relation between the two. Yes, it's possible to use ASCII escapes to create lots of special effects; heck, I used to do that myself, back in the early '80s for BBS "post-login" screens. I just seriously doubt that this hack from 25 year ago is still in any way "cool", or even interesting - and this is filtered through the fact that it's guaranteed to break on some percentage of terminals.

As a side note, I suspect that 'getty' replicates a fair amount of 'tput' functionality, since it has to deal with terminals at that level.

[[[[[[Thomas]]]]]] - You probably wouldn't, which is why the initscript in Gentoo and the like waste massive amounts of code in adding nice pretty colours which last for lesss than ten seconds on the screen, provide next to no manner in terms of usefulness and will fall over and die spectacularly on different terminals. Is this gain worth it? Of course not.

[[[[[[[Mahesh]]]]]]] - I was thinking of "private" use... in /etc/issue not /etc/issue.net?

[[[[[[[[Thomas]]]]]]]] - That depends on your distribution.

[[[[[[[[[Mahesh]]]]]]]]] - Arrgh! I don't seem to get my idea thru to you. See, when you feel like creating an /etc/issue that'll emit your '\l' getty escape codes using so-called hard-coded escape colour codes like "^[[33;1m\l^[[0m", how can tput come to the rescue? Like you code up a shell script to o/p (err.. output, sorry) a /tmp/issue that'll have correct escape codes querried form the terminfo database.

[[[[[[[[[[Kapil]]]]]]]]]] - As Ben pointed out you can indeed do that. For example, you can have /etc/default/issue which is used/parsed by /etc/init.d/generate_issue to generate the "real" /etc/issue by using the terminal settings appropriate to /dev/console (which may or may not be a serial console). Beware that this /etc/issue will break on some terminals that are "getty"-ed via serial line.

[[[[[[[Mahesh]]]]]]] - like it'd be useful to have ^[[31;1m\n^[[0m to see your hostname in bright red or something...

[[[[[[[[Thomas]]]]]]]] - Again, can be achieved via tput (you've been shown examples of this.)

[[[[[[[[[Mahesh]]]]]]]]] - Can tput escape codes be interpretted by getty?

[[[[[[[[[[Kapil]]]]]]]]]] - That "getty" only understands escape codes in /etc/issue instead of a generic description like that given by terminfo or slang is surely a fault of "getty"! Hack "getty" to fix this problem.

[[[[[[[[Ben]]]]]]]] - If I was going to do that - which I would not - I would generate '/etc/issue' via a script, with escape sequences created by 'tput'. However, the overriding problem that can't be handled at this level of interaction is that the console from which the user connects to your system is completely arbitrary - so there's no way for you to know what their correct escape sequences are. In other words, if (e.g.) a Solaris or a DOS user connects to you using a console that uses something different, all they're going to see is a bloody mess of square brackets and other line noise on their screen - and you can't prevent that from happening.

*This* is why it's a really bad idea to do those color tricks in '/etc/issue'. It's not because anybody's trying to prevent you from being cool; it's because you will inevitably make someone else's life harder by doing so.

[[[[[Mahesh]]]]] - I don't know, can u help?

[[[[[[Thomas]]]]]] - Nope, since I don't understand your question.

[[[[[[[Mahesh]]]]]]] - Like finding the codes o/p'd (err, outputted) by tput (maybe you wouldn't want it, but it could be found, isn't it?

[[[[[[[[Ben]]]]]]]] - Err, have you considered the word 'printed'? It's a perfectly good one, and causes no harm to anyone as far as I know.

[ In case you're still missing it: clarity, in a technical discussion, is far more important than saving a few keystrokes. WRITE OUT what you're trying to say to avoid confusion. ]

[[[[[[[[[[Ben]]]]]]]]]] - [ TAG forcibly re-CC'd. Mahesh, please leave the list in the CCs when replying; not doing so is considered rude. ]

[[[[[[[[[Mahesh]]]]]]]]] - Sorry, ed (warning: I may not be employed to write any public-readable stuff)

I wrote this, because I recently got a job offer from a leading magazine in India, to work as an in-house Linux Technical writer.

[[[[[[[[[[Ben]]]]]]]]]] - Much more of a reason to communicate clearly, then! Now, it's not just an adventure - it's your JOB...

(Not to carry the anvil too far, but some non-USians might miss it: the US Army's long-time advertising slogan was "It's not just a job - it's an adventure." These days, they've changed it to "An army of one" - presumably since anyone signing up these days will get ALL the "adventures" they never wanted.)

[[[Mahesh]]] - I even thought about fitting getopts in to select bold, underline, blink, etc.

[[[[Thomas]]]] - Depends what you want out of your script.

[[[[[Mahesh]]]]] - To help the newbies ease the complications...

[[[[[[Thomas]]]]]] - Pffft. Help them by teaching them, not dazzling their retenas with colours.

[[[[[[[Mahesh]]]]]]] - I was meaning those brainless twits who can't remeber all those escape codes. ("dazzling their retenas" -- is that a problem with my script?)

[[[[[[[[Thomas]]]]]]]] - You seem to be under the impression that your script is somehow lacking and isn't working. That's not true. Your script does work -- for some terminals. When you sent this script in to TAG, did think it was somehow not open for discussion? Constructive criticism such as the advice myself and Ben have been giving you is there to aid you; and not to pick any holes in your script.

[[[[[[[[[Mahesh]]]]]]]]] - Thomas (if I can call you by ur first name), my primary intention of writing the script was to find out the correct colour sequences to colour up the various getty control codes (\l, \m, \n) etc., in /etc/issue and for ${PS1}.

So I can take in any suggestions to make it more useful (beware, I'm working on v2.1 of the same) ...

useful, in the sense, anyone who feels like colouring their /etc/issue or /etc/motd can look up this script and decide on what to do (and what not to do.)

[[[[[[[[[[Kapil]]]]]]]]]] - As far as "motd" is concerned there is no short cut. You need to hack at "login" itself to make sure that it "knows" terminfo or slang in order that it can put out a colourful message of the day regardless of which terminal a user logs in from.

[[[Mahesh]]] - True, tput would have made it portable (a portable shell script???!!!).

[[[[Thomas]]]] - Careful -- the shell script itself already isn't portable (you use 'seq' which isn't on every system, think *BSD for instance), but it's portable in the sense that tput(1) is always guaranteed to work, where hard-coding escape sequences might not.

There's nothing wrong with your script, I was merely pointing out that tput(1) is by far the "better" choice when you wish to use colour for things such as prompts and the like.

[[[[[Mahesh]]]]] - Hey, I think the escape codes are handled by Linux console drivers, so how can they fail to work. At least thats what I understood after reading the man pages.

[[[[[[Thomas]]]]]] - No, they're handled by terminals which read the terminfo and termcap databases.

[[[[[[[Mahesh]]]]]]] - I think I misinterpreted what was in console_codes(4), can u correct me? I thought it said something like the Linux display driver handled the escape codes. Thanks.

[[[[[[[[Thomas]]]]]]]] - No, it's the terminal that interprets them. Note from console_codes(4):

It  is  generally not good practice to hard-wire terminal controls
into programs.  Linux supports a terminfo(5) database of terminal
capabili- ties.   Rather than emitting console escape sequences by hand,
you will almost always want to use a terminfo-aware screen  library  or
utility such as ncurses(3), tput(1), or reset(1).

[[[[[[[[[Mahesh]]]]]]]]] - Thanks, I might have missed it.

And one more thing, you don't mind me using this in my private /etc/issue, right? At least, decapitation is not an option for someone using hard coded escape sequences!

Thanks for ur time...

[[[[[Mahesh]]]]] - does the Linux cosole drivers in anyway deal with the terminfo/termcap facilities?

[[Mahesh]] - Ben, I'm happy I got a mail from LG. Thanks for mailing in..

[Quoting Ben]:

  > colsel () # Color selector - iterates through a $TERM's color choices 
  > { 
  > trap 'reset' 0 # Reset on exit

I didn't think of a trap ... sue me ;-)

[[[Ben]]] - I found that unless I used it, 'Ctrl-C'ing out of the script would often leave the xterm in some weird color mode. NOT a desirable outcome. :)

[[Mahesh]] - [Quoting Ben]:

  > > Well, the standard disclaimer applies. Any suggestions and corrections
  > > are welcome (all flames to /dev/null).
  > 
  > It may already be filled up; that's where we're sending ours...

I have written a script that will empty /dev/null the same way with /tmp at each boot time!!! :-D

[[[Ben]]] - Let me guess:

  # Save the old one - just in case, y'know... 
  ext=0
  while [ -f foo.$ext ]; do ((ext+=1)); done;
  mv /dev/null /dev/null.$ext 

  # Who needs that weird 'mknod' thing, anyway?
  # And by the way, why is my root partition all filled up? 
  touch /dev/null 

:)

[[Mahesh]] - [Quoting Ben]:

    > > > PS: If you want any help, do an "nl -ba colors.sh". ;-) 
    > > 
	> > I've always found 'vi' to be just as useful. :)

But do vi show line numbers (at default)

[[[Ben]]] - It does if you've got 'set nu' in your .exrc file.

[Kapil] - The discussion on pretty-printing of files naturally led to the discussion in the Unix-Hater's book (UHB) on how curses and terminfo are "twisted" technologies. According to the UHB the "right" way is that files should not be "streams of bytes" but should be "objects" which should (in particular) encode all pretty-printing information.

Interesting ... wonder if there is an implementation.

[[[[[[[[[Mahesh]]]]]]]]] - PS: Ben, I shoulda said this earlier: I learnt (well, most of) shell scripting thru a guide someone printed occassionaly in LG (I think his name was Ben Okopnik or something). It was a great guide. So you know whom to blame for my shell mistakes (OK, not escape-sequence mistakes ;) )

[[[[[[[[[[Ben]]]]]]]]]] - [laugh] My sins come back to haunt me. I'm glad that you found it useful, Mahesh (although I specifically remember using 'tput' in that series for "jazzing up" output in programs.)


Adding bogofilter to Exim4: ""doesn't work"".

Thomas Adam (thomas at edulinux.homeunix.org)
Fri Jun 2 06:03:09 PDT 2006

Answered by: Ben, Neil, Rick, Thomas

Hello,

At the moment my anti-spam measures are sub-optimal. The filtering of spam happens at fetchmail time via procmail into ~/Mail/spam. This used to be OK a few years ago as my address was relatively unknown. But now I must get at least anything between 40 - 80 spam emails a day. It's driving me nuts, so I figured it's time to do something at SMTP time.

I'm running Exim4 from Debian stable (which by definition means it's the Debianised configuration of Exim -- something I am still undecided as to whether this is a good idea or not). From my own knowledge about Exim and how emails are relayed at SMTP time, I think I understand how this all fits in, but I'll just make sure, as I don't understand why I get the error I do, and whether there's anything else I ought to be doing. (Rick, this is surely one for you... :P)

I'm not using the whole "split-file" configuration thing which Debian seems to like. Instead, if I want to make any changes, I just edit /etc/exim4/exim4.conf.template . Now, into that file, I have added two things. The first thing is the router. I suppose this ought to come after the DNS stuff. Hence I added:

bogo_router:
 domains = +local_domains
 no_verify
 condition = ${if !eq {$received_protocol}{bogodone} {1}{0}}
 driver = accept
 transport = bogo_transport

The corresponding transport for that, looks like:

bogo_transport:
 driver = pipe
 command = /usr/sbin/exim4 -oMr bogodone -bS
 use_bsmtp = true
 headers_add = X-Bogofilterd: true
 transport_filter = /usr/bin/bogofilter -d /etc/bogofilter -l -p -e -u
 return_fail_output = true
 group = mail
 user = mail
 home_directory = "/tmp"
 current_directory = "/tmp"
 log_output = true
 return_path_add = false

I've added that near the end of the template file -- I don't believe the position of the transport really matters. Indeed, having done that and running:

sudo update-exim4.conf && sudo /etc/init.d/exim4 reload

... produces no errors. However, sending a test email from my gmail account to ''thomas at edulinux.homeunix.org'' results in the following error:

2006-06-02 04:54:13 1Fm0jl-0006a4-8V ** thomas at edulinux.homeunix.org
R=bogo_router T=bogo_transport: Child process of bogo_transport transport
returned 2 from command: /usr/sbin/exim4

... indeed, the above comes from running eximon. Despite that error message eximon seems to think delivery has happened. This may indeed be the case but shortly after "delivery", the message is frozen, and the MAILER-DAEMON helpfully sends me back an email to my gmail account saying:

An error was detected while processing a file of BSMTP input.
The error message was:

 500 Command unrecognized

The SMTP transaction started in line 0.
The error was detected in line 1.
The SMTP command at fault was:

... so I've obviously balled something up, somewhere. Googling around a bit seems to reveal the configuration as I have it at the moment works for other people. I can't see in this situation what's wrong. I'm not even sure I know/understand what "BSMTP" is. I tried setting that to false in the transport section for bogofilter, but that was, err, clearly the wrong thing to do.

Can anyone else see what I've not done?

One of the reasons I'm not using the Exim-SA package or spamassassin in general is because it's a pig of a program and reduces poor fluffy (a P166 that acts as the kitchen sink) to a mere crawl. Since bogofilter seems to work just fine on my workstation, and that other people have seemingly integrated it into Exim makes me want to try the same.

Thanks in Advance.

-- Thomas Adam

[Ben] - Good idea; I'm slightly envious, given Rick's detailed expositions on it here. I can't implement the same thing due to lack of good network connectivity; that is the one requirement I do not meet for doing my my own SMTP.

I might have a little something to contribute, too - I've never done what you're talking about, but I've played with Exim extensively via its conffiles.

First, I'd do some general troubleshooting as described in the Exim manual (/usr/share/doc/exim/spec.txt.gz on my machine; I'm still using Exim 3, though.) Section 4.15, "Testing", covers all the basics: 'exim -bV' for a static test, 'exim -v -bt <localuser>' for local and 'exim -v -bt <remoteuser>' for remote delivery, "SMTP conversation", i.e.

  exim postmaster at your.domain
  From: user at your.domain
  To: postmaster at your.domain
  Subject: Testing Exim

  This is a test message.
  ^D

for a full delivery test, etc. Then there's my personal favorite, 'exim -d9 -M <message-id>' - turning on max_debug and vgrepping through lots of log messages... seriously, that could be very useful. Especially if you turn it on in your transport section and try that Gmail interaction again.

Give it a shot and let's see what you get!

[[Thomas]] - Hey, Ben -- replying via webmail, since I'm leaving my exim process on my server running in debug mode, and with the bogo stuff turned on, nothing gets delivered. :P

[[[Ben]]] - That's the problem with mail problems - the thing with the problem is the thing you need to fix the problem!

[[Thomas]] - I have heard rumours that Exim works quite well in disconnected setups like yours, although I can't say how good/bad that is.

[[[Ben]]] - It works well WRT delivery, etc.; it's bloody awful if you want to run SMTP-time checking - since that requires a reasonably fast network connection, otherwise the per-mail processing time goes into orbit.

[[Thomas]] - Ok -- the static tests fail. In fact, any test with the bogo-filter stuff enabled just barfs (same error as shown in the file in the URL quoted further down, BTW).

[[[Ben]]] - Probably not an Exim configuration problem, then.

[[Thomas]] - [quoting Ben]

'exim -d9 -M <message-id>' - turning on max_debug and vgrepping through

OK -- it seems that the "-d9" idea isn't applicable on Exim 4. Looking in its manpage, there seems to be "-d all" which sounded promising, but which just held up Exim and wouldn't do anything after that. So I used the following command:

 sudo exim4 -d -M 1Fn2PM-0007BB-EV 2<&1 | tee /tmp/exim.out

... that message ID is of an email I had sent on the localhost -- the `same error happens from anywhere. I shan't paste the output from that command in here directly. Instead you can see the output here.

[[[Ben]]] - I note a couple of questionable log messages there - e.g.,

internal_search_find: file="/etc/aliases"
  type=lsearch key="n6tadam"
file lookup required for n6tadam
  in /etc/aliases
lookup failed
expanded: 
file is not a filter file
parse_forward_list: 
system_aliases router declined for n6tadam at edulinux.homeunix.org
which may or may not be of concern for you; however, the important bits are obviously toward the end:
calling bogo_router router
bogo_router router called for n6tadam at edulinux.homeunix.org
  domain = edulinux.homeunix.org
set transport bogo_transport
queued for bogo_transport transport: local_part = n6tadam
domain = edulinux.homeunix.org
  errors_to=NULL
That last line should probably be redefined for better troubleshooting.
  domain_data=NULL localpart_data=NULL
routed by bogo_router router
  envelope to: n6tadam at edulinux.homeunix.org
  transport: bogo_transport
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
After routing:
  Local deliveries:
    n6tadam at edulinux.homeunix.org

So far, so good.

  Remote deliveries:
  Failed addresses:
  Deferred addresses:
search_tidyup called
>>>>>>>>>>>>>>>> Local
deliveries >>>>>>>>>>>>>>>>
--------> n6tadam at edulinux.homeunix.org <--------
locking /var/spool/exim4/db/retry.lockfile
locked /var/spool/exim4/db/retry.lockfile
opened hints database /var/spool/exim4/db/retry: flags=0
dbfn_read: key=T:n6tadam at edulinux.homeunix.org
no retry record exists
search_tidyup called
changed uid/gid: local delivery to n6tadam <n6tadam at
edulinux.homeunix.org> transport=bogo_transport
  uid=8 gid=8 pid=28344
  auxiliary group list: <none>
  home=/tmp current=/tmp
set_process_info: 28344 delivering 1Fn2PM-0007BB-EV to n6tadam using
bogo_transport
direct command:
  argv[0] = /usr/bin/bogofilter
  argv[1] = -d
  argv[2] = /etc/bogofilter
  argv[3] = -l
  argv[4] = -p
  argv[5] = -e
  argv[6] = -u
direct command after expansion:
  argv[0] = /usr/bin/bogofilter
  argv[1] = -d
  argv[2] = /etc/bogofilter
  argv[3] = -l
  argv[4] = -p
  argv[5] = -e
  argv[6] = -u

Have you double-checked to make sure that

'/usr/bin/bogofilter -d /etc/bogofilter -l -p -e -u'

actually works? I'd do it with the options spelled out, exactly as Exim sees it.

bogo_transport transport entered
direct command:
  argv[0] = /usr/sbin/exim4
  argv[1] = -oMr
  argv[2] = bogodone
  argv[3] = -bS
direct command after expansion:
  argv[0] = /usr/sbin/exim4
  argv[1] = -oMr
  argv[2] = bogodone
  argv[3] = -bS

Ditto here. Obviously, these two are hooked up via a pipe - so that's the manual process to try.

Writing message to pipe
writing data block fd=10 size=13 timeout=3600
writing data block fd=10 size=40 timeout=3600
writing data block fd=10 size=5 timeout=3600
process 28347 running as transport filter: write=11 read=12
added header line(s):
X-Bogofilterd: true---
process 28348 writing to transport filter
copying from the filter
set_process_info: 28346 reading output from |/usr/sbin/exim4 -oMr bogodone
-bS

Just for grins... what does Exim return when run that way? '$?' should tell you.

writing data block fd=11 size=2252 timeout=3600
waiting for filter process
filter process returned 3
waiting for writing process
end of filtering transport writing: yield=0
errno=-24 more_errno=3

Whoops. I wonder what that means. You might want to Google for it... incidentally, I just noticed that my '/usr/share/doc/exim' directory contains a discussion of filters and how to use them. Again, this is Exim3 - but you might have a parallel document that describes the specifics, and might be worth checking out.

search_tidyup called
bogo_transport transport returned PANIC for n6tadam at
edulinux.homeunix.org
post-process n6tadam at edulinux.homeunix.org (9)
LOG: MAIN
  <n6tadam at edulinux.homeunix.org>: bogo_transport transport
output: An
error was detected while processing a file of BSMTP input.
LOG: MAIN PANIC
  == n6tadam at edulinux.homeunix.org <thomas at
edulinux.homeunix.org>
R=bogo_router T=bogo_transport defer (-24): Filter process failure
2006-06-05 01:17:30 1Fn2PM-0007BB-EV == n6tadam at edulinux.homeunix.org
<thomas at edulinux.homeunix.org> R=bogo_router T=bogo_transport
defer
(-24): Filter process failure
2006-06-05 01:17:30 1Fn2PM-0007BB-EV == n6tadam at edulinux.homeunix.org
<thomas at edulinux.homeunix.org> R=bogo_router T=bogo_transport
defer
(-24): Filter process failure
exim: could not open panic log - aborting: see message(s) above

Hmm, looks like another place where you could get troubleshooting info. Do you have a '/var/log/exim/paniclog' file on your system?

So, I'd run through all of the above stuff and see what shakes out. If you don't get definitive results from the above (and I'm sorta betting on testing the piped progs), then feel free to describe what you've got and we'll bang on it some more. :)

[[Thomas]] - I'm not turning up many hits on Google which sheds much light on this -- despite triple checking the existing filter for bogo_filter matches the general concensus which purportedly works on other's machines; and it does match.

[Neil] - [Quoting Thomas]:

  > 2006-06-02 04:54:13 1Fm0jl-0006a4-8V ** thomas at edulinux.homeunix.org 
  > R=bogo_router T=bogo_transport: Child process of bogo_transport transport 
  > returned 2 from command: /usr/sbin/exim4

Someone else had the same problem. I'm not sure that the answer he got was very useful :-(

http://www.exim.org/mail-archives/exim-users/Week-of-Mon-20030811/msg00247.html

FWIW according to http://bogofilter.sourceforge.net/man_page.shtml return values are "0 for spam; 1 for non-spam; 2 for unsure ; 3 for I/O or other errors.

If both -p and -e are used, the return values are: 0 for spam or non-spam; 3 for I/O or other errors."

It's interesting that you seem to be using -p and -e, but still get a return value of 2.

According to http://exim.org/exim-html-4.62/doc/html/spec_html/ch29.html#id2641142: "If the command exits with a non-zero return code, the delivery is deemed to have failed, unless either the ignore_status option is set (in which case the return code is treated as zero), or the return code is one of those listed in the temp_errors option, which are interpreted as meaning “try again later”."

So the problem would appear to be the return code of 2 from bogofilter, which AFAICS you should not be getting.

[Rick] - [Quoting Thomas]:

  > I'm running Exim4 from Debian stable (which by definition means it's the 
  > Debianised configuration of Exim -- something I am still undecided as to 
  > whether this is a good idea or not).

My middle initial "A" honestly doesn't stand for "Ambivalent", but I have ongoing doubts about that too -- and I rely on it. ;->

  > From my own knowledge about Exim and 
  > how emails are relayed at SMTP time, I think I understand how this all 
  > fits in, but I'll just make sure, as I don't understand why I get the 
  > error I do, and whether there's anything else I ought to be doing. 
  > (Rick, this is surely one for you... :P) 

OK, but I'm a bit of a seat-of-my-pants flier, when it comes to MTA configuration. I've never had a machine set up for experimentation; anything I attempt is on a production mail system, where people call and scream at me if I break anything. Accordingly, opportunities for tinkering have been few, and cautious.

  > I'm not using the whole "split-file" configuration thing which Debian 
  > seems to like. Instead, if I want to make any changes, I just edit 
  > /etc/exim4/exim4.conf.template  .

That's one of the things on which I have greatest ambivalence, actually. The split-file option is best for long-term maintenance, in that the system can incrementally upgrade the parts you haven't touched, while leaving alone others. On the other hand, the single-file option's significantly easier to grok.

  > Now, into that file, I have added two 
  > things. The first thing is the router. I suppose this ought to come 
  > after the DNS stuff. Hence I added: 
    
  >   bogo_router: 
  >   domains = +local_domains 
  >   no_verify 
    
  >   condition = ${if !eq {$received_protocol}{bogodone} {1}{0}} 
  >   driver = accept 
  >   transport = bogo_transport 

Hmm, there used to be a third-party modification for Exim4 called Exiscan (http://duncanthrax.net/exiscan-acl/). I see that it's now merged into Exim4 mainline code -- but you might double-check that your chosen Debian package includes that functionality. E.g., it might be that package exim4-daemon-light doesn't, but exim4-daemon-heavy does.

The Exiscan patch added support for a broad range of content-scanning (virus, spam, other), while Marc Merlin's SA-Exim patch is specialised for SpamAssassin integration and (I think) SA-specific. Exiscan's most recent versions (dubbed Exiscan-ACL) work by adding additional functions to Exim4's DATA ACL, while also providing an entirely new ACL (acl_smtp_mime) called for each MIME part in a message. You might want to look into that approach, rather than attempting to hack transports and routers.

  > One of the reasons I'm not using the Exim-SA package or spamassassin in 
  > general is because it's a pig of a program and reduces poor fluffy (a P166 
  > that acts as the kitchen sink) to a mere crawl.

However, note that one of the key aims of an SA-Exim/SpamAssassin setup is to use SA as little as possible. If you start out (as I did) with J.P. Boggis's EximConfig set of default ACLs / configuration files, most of the heavy lifting is performed not by SA but rather by the Exim4 ACLs that handle the mail first, and reject most of the junkmail.


Starting to explore the Linux kernel code

Deepak Mishra (mishradk at gmail.com)
Thu Jun 8 15:05:15 PDT 2006

Answered by: Deepak, John, Lew, Marty, Ramon, Rick, Thomas

Hello Everyone ( Answer Gang), I have gone through the basics of OS and now trying to explore and understand the linux kernel code. I am confused from where I should start. Just to make my life easier, I am thinking to to document the function calls from the login prompt to bash or any shell prompt.

Then I want to go in reverse direction and find out how I came up to a log in prompt.

As at log in prompt I see <hostname> login: so I tried to grep for the "login:" in .c files. but really could not find the the proper location in the source code. I have a 2.6.15-01.2054_FC5 source code.

Am very very interested to understand this code. I understand C language quite well.

I have 3 questions.
1. Is this approach to understand the linux kernel code all right?
2. If any one can give me some hint where I can find the login related code or where I should start .. I will be very glad.
3. Is there any documentation available which I can use as reference?

thank you verry much. Regards Deepak

[Ramon] - Although it sounds like a nice idea to start with linux kernel source code, logging in doesn't really use the kernel code much. (Or it does but in a very different way then you seem to assume)

You might want to start by having a look at the linux bootproces and understanding the role that the kernel fills in this proces.

Recently someone published a nice overview of the boot process on the ibm developerworks site

Additionally I always found the linux from scratch project an excellent resource on the ecosystem that the linux kernel + the (GNU) userland tools & programs really is.

You can find it here: http://www.linuxfromscratch.org/

  > I have 3 questions.
  > 1. Is this approach to understand the linux kernel code all right?

No, see above. Basically the kernel handles access to the hardware resources in an uniform way and offers hooks for user-space programs to get access to these resources. You will find that the login process is handled by a program that calls the kernel to get access to the appropriate resource (filesystem, screen, keyboard, serial line, whatever)

  > 2. If any one can give me some hint where I can find the login related
  > code or where I should start .. I will be very glad.
  > 3. Is there any documentation available which I can use as reference?

See above.

If you have a spare machine lying around somewhere that you can use I can really recommend linux from scratch. It will give you an excellent insight in how what most people refer to as linux is actually build from a number of different components, among them a kernel. It doesn't have to be a high-specced machine, I originally build a linux from scratch on an old intel 386 machine.

Hope you enjoy the ride ;-)

[Lew] - You probably should read the "From PowerUp to Bash Prompt HOWTO" which covers that topic quite well

You might also want to read up on Operating System design and implementation, as the background will help you understand what's going on. A /really/ good book is "Operating Systems: Design and Implementation" by Andrew Tanenbaum and Albert Woodhull (ISBN: 0-13-142938-8, Prentice Hall). You might also want to study how Minix was built (Minix is a unix-like OS built as a teaching tool, and is documented in OSD&I). That might give you a good background on how things work in the Unix and Linux world, and where you might reasonably find the facilities and tools implemented.

  > As at log in prompt I see  <hostname> login:
  > so I tried to grep for the "login:" in .c files. but really could not
  > find the the proper location in the source code.

The login prompt isn't part of the OS kernel. It is actually generated by a user program (getty, iirc). To see it, you would have to grep through the code for getty.c, which I bet you haven't installed (since the userspace app source code is rarely installed on general machines).

  > I have a 2.6.15-01.2054_FC5 source code.
  > 
  > Am very very interested to understand this code. I understand C
  > language quite well.
  > 
  > I have 3 questions.
  > 1. Is this approach to understand the linux kernel code all right?

It is one way to do it. It might not be the best way if you don't already know what Unixish operating systems must do to start up.

  > 2. If any one can give me some hint where I can find the login related
  > code or where I should start .. I will be very glad.

Look at the source code for the getty, login, and passwd user applications. This will /not/ be in the Linux kernel source code libraries.

  > 3. Is there any documentation available which I can use as reference?

See above.

[John] - Just yesterday I found an online copy of the historical book by J. Lions, "A Commentary on the Sixth Edition Unix Operating System". The link can be found under the heading "external links" on the wikipedia page for "lions book".

Although the material is quite dated, it is still highly regarded as a reference for basic unix OS principles, so you might find it of interest.

[[Thomas]] - I still find the Unix Haters book a much "better" authority on the OS. :P

[[[Deepak]]] - Thank you all for your valuable suggestions. After listening from you people i have started building my own Linux system based LFS.

[[[John]]] - Ah, just as I suspected - you are from the dark side ... :)

[[[Marty]]] - I found the UHG

[[[[Thomas]]]] - Was it good? Did you read it? I can't help but think there was more to this email than just that sentence above, initially. :P

[[[[[Rick]]]]] - If anyone hasn't yet read this 1994 collection of Unix discontents, it's definitely worth the trouble, if only because the ranting is sometimes inventive and literate: http://research.microsoft.com/~daniel/unix-haters.html

Dennis Ritchie's comment in his "anti-forward" is worth noting: "You claim to seek progress, but you succeed mainly in whining." That's the lesser problem with the book (particularly in the parts written by LISP Machine and ITS fans whose preferred environments early Unix replaced): It's just a miscellany of sundry people's personal piques and resentments. The larger problem is that those complaints, though they were sometimes reasonable if (mostly) exaggerated at the time they were posted to the unix-haters mailing list -- late 1980s to early 1990s -- just about all seemed already quaint by the time of publication.

In fact, the key ingredient required for fixing those antique problems was open-source licensing and source code access, which was arriving in the form of the BSD family and Linux precisely as the book was being published in 1994.

Chris Browne's review of the now-publicly-available text is here: http://linuxfinances.info/info/unixhaters.html

[[[Marty]]] - I found the Unix haters guide made some valid criticisms, but I knew some of the people on the mailing list -- and found they're vociferousness humorous.

[[[[John]]]] - Yes, most of it does seem to have been written in a humorous vein, some of it very tongue-in-cheek. I haven't yet read it entirely though.

[[[Marty]]] - Their major criticism was "Unix is too complicated" which translates into Doug Gwyn's remarks (which are so true) "GUIs make simple things simple and difficult things impossible".

The criticism boils down to "I only want to do simple things, I can't learn how to do complicated things". If the GUI doesn't let me, I don't need to...so this really means "I'd rather spend a week doing writing code to do in awk/sed (back then) in 5 minutes..."

[[[[John]]]] - A lot of it was written in the early days of history of the OS family, with several mentions of Lisp Machines, etc. It doesn't seem to reflect a lot of the advances made during the past 12 years or so:

    Did users want the operating system where bugs didn't get fixed? Not
    likely. Did users want the operating system with a terrible tool set?
    Probably not. Did users want the OS without automatic command
    completion?  No. Did users really want the OS with a terrible and
    dangerous user interface?  No way. Did users want the OS without memory
    mapped files? No.  Did users want the OS that couldn't stay up more
    than a few days (sometimes hours) at a time? Nope. Did users want the
    only OS without intelligent typeahead? Indeed not. Did users want the
    cheapest workstation money could buy that supported a compiler and
    linker? Absolutely.  They were willing to make a few sacrifices.

[[[Marty]]] - BTW, the latest WSYSIWIG editor I enjoyed using was BravoX on the altos.


Message sending failure

Martin Hooper (martinjh at blueyonder.co.uk)
Mon Jun 19 09:00:52 PDT 2006

Answered by: Martin, Rick

Any reason this might have happened? Not sure whether it could be a problem at my ISP or at the other end..?

If it could be my ISP then I'll mention on their support groups.

-------- Original Message --------
Subject: Mail delivery failed: returning message to sender
Date: Mon, 19 Jun 2006 16:48:51 +0100
From: Mail Delivery System <Mailer-Daemon at blueyonder.co.uk>
To: martinjh at blueyonder.co.uk

This message was created automatically by mail delivery software.

A message that you sent could not be delivered to one or more of its
recipients. This is a permanent error. The following address(es) failed:

   tchung at fedoranews.org
     SMTP error from remote mail server after MAIL 
FROM:<martinjh at blueyonder.co.uk>:
     host fedoranews.org [64.34.165.170]: 550 5.7.1 Access denied

------ This is a copy of the message, including all the headers. ------

Return-path: <martinjh at blueyonder.co.uk>
Received: from [172.23.170.146] (helo=anti-virus03-09)
	by smtp-out5.blueyonder.co.uk with smtp (Exim 4.52)
	id 1FsLze-0003tl-GW
	for tchung at fedoranews.org; Mon, 19 Jun 2006 16:48:50 +0100
Received: from [62.30.33.77] (helo=[192.168.0.2])
	by asmtp-out2.blueyonder.co.uk with esmtp (Exim 4.52)
	id 1FsLzd-00033G-Ni
	for tchung at fedoranews.org; Mon, 19 Jun 2006 16:48:50 +0100
Message-ID: <4496C760.8070803 at blueyonder.co.uk>
Date: Mon, 19 Jun 2006 16:48:48 +0100
From: Martin Hooper <martinjh at blueyonder.co.uk>
User-Agent: Thunderbird 1.5.0.4 (Wind0ws/20060516)
MIME-Version: 1.0
To:  tchung at fedoranews.org
Subject: Fedora Security Bulletins
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Tom,

Having returned to Fedora after having used Kubuntu for a while I was
wondering if there was a security update announce list I could subscribe
to??

Keep up the good work on Fedora News...

Martin

[Rick] - Cutting to the chase and answering your question, the "reject" DSN (Delivery Status Notification) you received was reported by host fedoranews.org, IP 64.34.165.170, your intended destination, which for not-very-well-explained reasons has turned surly and is declining mail (yours, at least; probably others', as well). However, let's trace the progress of your message, so you can see how to interpret the DSN and the headers in your attached mail copy:

First, consider the DSN text itself:

>    tchung at fedoranews.org
>      SMTP error from remote mail server after MAIL
>      FROM:<martinjh at blueyonder.co.uk>:
>      host fedoranews.org [64.34.165.170]: 550 5.7.1 Access denied

The "host fedoranews.org [64.34.165.170]" part identifies what machine is notifying you. It's declining mail you addressed to "tchung at fedoranews.org". The DSN payload, which are the error code and explanatory text (for small values of "explanatory" in this particular case) are the final bit:

> 550 5.7.1 Access denied

Let's break that down. It parses as:

o SMTP error code: 550 o SMTP extended error code: 5.7.1 o SMTP error text: Access denied.

"550" is the classic (most common) permanent-failure SMTP error code, though there are others. The 5xx category are permanent failure reports, i.e. "this is definitive, so there's no point in re-attempting delivery" -- in contrast to the 4xx temporary-failure errors you will often see, such as those ending with "Warning: message still undelivered after 4 hours. Will keep trying until message is 5 days old", and the 2xx series that indicate successful delivery.

I can't count the number of times people have telephoned me during my mail server's scheduled downtime, complaining they got a "bounce message" when they tried to mail me -- only to find out that the diagnostic text, which they didn't bother to read, very clearly said (paraphrasing) "I've been having some difficulty getting this message through; it may be just a glitch, and there are still five days of delivery attempts left, but I thought you should know."

A fair amount of thought has gone into making (most) DSNs informative; it's a pity so many people can't be bothered to read them _at all_.

"550" and all the other SMTP error codes were defined in Jon Postel's classic RFC821 document, in 1982. "550" is explained there as being intended for "Requested action not taken: mailbox unavailable [E.g., mailbox not found, no access]" situations -- wording broad enough to serve as a catch-all for most permanent delivery failures, other than 50x SMTP-command syntax errors, 551 "User not local" redirects, 552 "exceeded storage allocation" errors, 553 "mailbox syntax incorrect" errors, and a vague-sounding 554 "transaction failed" that most often means "destination host is so overloaded that it's refusing some mail as a coping strategy".

But don't forget my larger point that you needn't know that, to read the error messages: Generally, you just read the error text, and the reason's _right there_.

"Access denied" in this case is, sadly, pretty delphic, and could mean quite a lot of things, I suppose.

That brings us to "5.7.1": RFC1821's definition of the SMTP standard in August 1982 was, by a decade and a half later, wildly successful by any measure, but among its nagging design defects was the limitations in its set of error codes. Many were just not specific enough, and there were very few unassigned code numbers left to fill in gaps. Therefore, "subcodes" were defined in January 1996's RFC1893, and follow a three-part numerical format: "class.subject.detail".

The leading "class" tuple uses the same numerical convention the earlier SMTP error codes did:

2.X.X = success
4.X.X = temporary failure
5.X.X = permanent failure

Each of these is further specified by "subject" tuples for the middle value:

X.0.X = this error concerns: other/undefined
X.1.X = this error concerns: sender or recipient address syntax/validity
X.2.X = this error concerns: recipient mailbox issues
X.3.X = this error concerns: recipient system problems
X.4.X = this error concerns: network or routing problems
X.5.X = this error concerns: failure somewhere in the mail delivery
protocol
X.6.X = this error concerns: failure involving content-handling
X.7.X = this error concerns: security or policy failures

I could give a complete listing of the "detail" permutations (the third tuple), but it'd be a bit long -- 49 entries -- because meaning differs based on what values the other two tuples have. The full list is in RFC1893, but the one relevant to your mail is:

X.7.1   Delivery not authorized, message refused

          The sender is not authorized to send to the destination.
          This can be the result of per-host or per-recipient
          filtering.  This memo does not discuss the merits of any
          such filtering, but provides a mechanism to report such.
          This is useful only as a permanent error.

So, basically, this string you got -- "550 5.7.1 Access denied" amounts to: fedoranews.org thinks that you, your ISP, or the particulars of your message shouldn't not be allowed to use it to send mail to mailbox "tchung", for reasons basically unspecified but vaguely security or filtering-related. Assuming you haven't done anything to get Tom Chung's knickers into a bunch ;-%gt; , possibly he has a misconfigured or severely trigger-happy antivirus or antispam system? Or there's bad blood between fedoranews.com and Blueyonder?

Along those lines, the error was triggered at exactly the point where your ISP's outbound SMTP server said to fedoranews.org

    MAIL FROM:<martinjh at blueyonder.co.uk>

Looks like something at fedoranews.org doesn't like specifically your mailbox -- or your ISP. Either way, it seems a bit ill-tempered, at best.

You can follow the actual literal progress or your message, up to the point where fedoranews.com rejected it, in the pair of Received headers sent back to you:

> Received: from [172.23.170.146] (helo=anti-virus03-09) > by smtp-out5.blueyonder.co.uk with smtp (Exim 4.52) > id 1FsLze-0003tl-GW > for tchung at fedoranews.org; Mon, 19 Jun 2006 16:48:50 +0100 > Received: from [62.30.33.77] (helo=[192.168.0.2]) > by asmtp-out2.blueyonder.co.uk with esmtp (Exim 4.52) > id 1FsLzd-00033G-Ni > for tchung at fedoranews.org; Mon, 19 Jun 2006 16:48:50 +0100

Those are most-recent first, so you actually start reading at the bottom and go up:

1. MTA "asmtp-out2.blueyonder.co.uk" (presumably, your ISP's outbound SMTP host) received a message stream from IP address 62.30.33.77 (presumably, your workstation), addressed to tchung at fedoranews.org asmtp-out2 then passed the message along to the next phase.

2. That next phase is apparently some sort of additional outbound-SMTP machine called "smtp-out5.blueyonder.co.uk" (maybe a Micr0s0ft-virus stripper?), which does the actual delivery to a mail exchanger for the "fedoranews.org" domain.

How does it know where such mail should go? We can simulate its thinking processes using the "dig" command: Mail should be dropped off at a host indicated in the domain's "MX" DNS records. If there are no such records, it should be dropped off using the DNS "A" record for the specified host+domain, as a fallback.

$ dig -t mx fedoranews.org +short
$
Null result on type "MX".  So, as mentioned, there's an automatic
fallback to "A" (regular forward lookup) records:

$ dig -t a fedoranews.org +short
64.34.165.170

3. So, smtp-out5.blueyonder.co.uk attempted delivery to "fedoranews.org" AKA IP address 64.34.165.170, where it ran into the aforementioned security or policy-based refusal.

There's no "Received" header for this third hop because, well, the mail was refused, after all.

Just as a word of caution, not all Received headers are honest: Spammers, computer criminals, and some malware have been known to attempt to inject bogus information or even one or more entirely fraudulent Received headers, in an effort to deceive analysts trying to track down responsible parties. In general, any information in an SMTP e-mail, including all Received headers, can be forged by any of the prior machines that handled the mail, and must logically be subject to doubt, except that the immediately prior MTA's IP address, before your own, is known with certainty -- because it was recorded by your MTA at the time of receipt.

Thus, if you were the sysadmin of smtp5-out.blueyonder.co.uk, and trusted your own machine (to not be root-compromised, etc.), then you could rest assured that this mail really did come from IP address 172.23.170.146 .

[Martin] - Thanks Rick - Very informative as usual... ;)

Talkback: Discuss this article with The Answer Gang

Copyright © 2006, . Released under the Open Publication license unless otherwise noted in the body of the article. Linux Gazette is not produced, sponsored, or endorsed by its prior host, SSC, Inc.

Published in Issue 129 of Linux Gazette, August 2006

<-- prev | next -->
Tux