So you are half correct, in a single photo you are getting better resolution at a lower altitude. however since you are going to be sticthing these photos together the software needs more data to accurately process and determine tie points. As @dragonflyAS alluded to i have hit on this topic in another post but ill gladly help clarify here. It might seem weird but we have to separate the detail in one photo from the overall quality of orthomosaics. The lower you fly the more detail in a smaller area you begin to capture (this is great for a single photo) but, it means that the photo itself actually contains less data over all (since it is capturing a smaller area). For high quality orthos flying higher means you are covering more area per a photo and therefore each photo contains more raw data. This is essential for the software to be able to accurately determine the tie points in each photo to be able to render a high quality ortho.
In general the higher you fly the better the overall ortho will come out. This is a balance game, i always suggest flying several test flights at various altitudes to find what works best for your needs. It is also important to remember not every job requires a hammer. Meaning while an ortho is great and cool it is not always the best tool for the job in some cases using the individual photos at lower altitudes may be what you need; in others flying higher with a great ortho will get you the results you want. The key is to be "product oriented" do what you need to do to be able to get the results you need for a client.
So in short... ironic i suppose... you can fly lower but that may not get the result you are looking for.