Usually the only tricky part of compiling from source is tracking down dependencies. The package manager does that for you normally but you’re not using the package manager when compiling from scratch. The actual building (even compiling a kernel) isn’t all that complicated.