Any software required to support the production or execution of application programs but which is not specific to any particular application.
System software typically includes an operating system to control the execution of other programs; user environment software such as a command-line interpreter, window system, desktop; development tools for building other programs such as assemblers, compilers, linkers, libraries, interpreters, cross-reference generators, version control, make; debugging, profiling and monitoring tools; utility programs, e.g. for sorting, printing, and editting.
Different people would classify some or all of the above as part of the operating system while others might say the operating system was just the kernel. Some might say system software includes utility programs like sort.