In his old age SardarPatel was so busy that he did not have a moment's rest. He had to tour extensively all over the country; this affected his health. Patel's health declined rapidly through the summer of 1950. He later began coughing blood, whereupon Maniben, his daughter, began limiting his meetings and working hours and arranged for a personalised medical staff to begin attending to Patel. The Chief Minister of West Bengal and doctor Bidhan Roy heard Patel make jokes about his impending end, and in a private meeting Patel frankly admitted to his ministerial colleague N. V. Gadgil that he was not going to live much longer.

Patel's health worsened after 2nd November, when he began losing consciousness frequently and was confined to his bed. He was flown to Mumbai on 12 December on advice from Dr. Roy, to recuperate as his condition deemed critical. The gathering at the Delhi airport to bid him farewell was unusually large; it included Nehru, Rajagopalchari, Rajendra Prasad and Menon. They all knew that they were seeing him for the last time. Patel was extremely weak and had to be carried onto the aircraft in a chair. In Bombay, large crowds gathered at Santacruz Airport to greet him. To spare him from this stress, the aircraft landed at Juhu Aerodrome, where Chief Minister B.G. Kher and Morarji Desai were present to receive him with a car, that took Vallabhbhai to Birla House.

After suffering a massive heart attack (his second), he died on 15 December, 1950 at Birla House in Bombay. He was then seventy-five. His cremation was planned at Girgaum Chowpatty, however this was changed to Sonapur (Now Marine Lines) when his daughter conveyed that it was his wish to be cremated like a common man in the same place as his wife and brother were earlier cremated. His cremation in Sonapur in Bombay, was attended by a 10 lakh strong crowd including Rajagopalachari, and President Rajendra Prasad.

In an unprecedented and unrepeated gesture, on the day after his death more than 1,500 officers of India's civil and police services congregated to mourn at Patel's residence in Delhi and pledged 'complete loyalty and unremitting zeal' in India's service.