Buffer Overflow attack happens when more data sent to a system than it's designed to handle. This kind of attack may happen through unsecured programming , say if a program copies an input buffer to an output buffer without verifying that the
size of the input buffer is less than the size of the output buffer it may cause a buffer overflow. When buffer overflow attack happens , if it overwrites the allocated memory from the stack area then it's a stack based buffer overflow and when it overwrites the heap area then it's heap based buffer overflow. This kind of attack may lead to a system crash.
There are several preventive measures to be taken against this attack. They are discussed below:
i) Disallowing executable code by adding a line like 'set noexec_user_stack=1' to the /etc/system file. Now if a program attempts to execute code on their stack it will be aborted with a core dump and a warning message will be displayed with the process name, PID and UID. This message can be logged by syslog. Enable it by adding another to the /etc/system file , 'set noexec_user_stack_log=1'. A reboot is necessary to make it work.
ii) Removing unnecessary compilers.
iii) Unknown programs must NOT be executed.
iv) Disabling unnecessary ports and services by commenting lines in /etc/inetd.conf file.
v) Applying patches from Sun website.
vi) Using ASET (Aotomated Security Enhancement Tool) in high mode. (Check out 'man aset').
