IB/hfi1: Adjust default eager_buffer_size to 8MB

IB/hfi1: Adjust default eager_buffer_size to 8MB

Performance analysis shows benefits for PSM2 in increasing eager buffer
size from 2MB to 8MB. The change has neutral impact on verbs.
Make change to the module parameter's default value. Allocation
ring down was verified to work with the larger buffer size.

Reviewed-by: Tadeusz Struk <tadeusz.struk@intel.com>
Reviewed-by: Mike Marciniszyn <mike.marciniszyn@intel.com>
Reviewed-by: Dennis Dalessandro <dennis.dalessandro@intel.com>
Signed-off-by: Tymoteusz Kielan <tymoteusz.kielan@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessandro@intel.com>
Signed-off-by: Doug Ledford <dledford@redhat.com>