Keys received treatment on her leg after dropping serve in the second set to give Venus Williams a 4-1 lead. After the medical timeout, she broke Williams' serve twice to level the set at 4-all, but Williams then captured the next two games to send it to a decider.
Keys was also behind 3-1 in the final set before breaking Williams' serve three times in a row to close out the match.
The match featured 12 service breaks, seven won by Keys, and 83 combined unforced errors.
The loss ended an encouraging 10 days for Venus Williams at Melbourne Park. She hadn't advanced to the quarterfinals of a Grand Slam since the US Open in 2010 and had struggled to recapture her earlier form after being diagnosed in 2011 with Sjogren's syndrome, an auto-immune disease that causes fatigue and joint pain.
But Williams is 9-1 to start the new year after capturing her season-opening tournament in Auckland, New Zealand, before the Australian Open.
"It already feels like a long season already, so many matches in a row," she said. "But it's a great start. Hopefully I'll be able to keep this level up."
Regardless of the quality of the error-strewn match, Keys was pleased to be through to the semis.