I've tried looking for this on here and on google but I haven't really found an answer to this specific question and I'm not sure if there's something - a term or something that I need to look for, so I apologize if this is a commonly asked question.
Either way, let's say for example that I'm building a website with many sections and I wanted these sections to be modular, so each section is it's own html file. So basically I could include this little module anywhere I want on the main html file or maybe I could simply include navbars and footers onto other HTML pages without having to rewrite lines of code.
Is this possible with just HTML alone?
Using HTML5 Import. It does have very limited support/browsers implementing it.
<link href="extern.html" rel="import" />
Another possibility as Pete highlighted is the use of iframes (although I would prefer not to use them).
The use of HTML5 Imports is highly discouraged as you can see on here.
Here's few notes taken from the above link:
MS Edge status: Under Consideration
Chrome status: Deprecated
Firefox status: not-planned
WebKit status: Not Considering
Firefox has no plans to support HTML imports though for now it can be enabled through the "dom.webcomponents.enabled" preference in about:config