This is an action initially begun by the Equal Employment Opportunity Commission [EEOC] against the defendant Federal Reserve Bank of Richmond. The defendant, chartered under the Federal Reserve Act,1 operates a branch in Charlotte, North Carolina, which provides (1) services to the member banks and the public in the Charlotte area in check collection, adjustment, and provision of cash and securities and (2) services to the United States Treasury and governmental agencies in handling savings bonds and government securities, including food stamp activities. In performing these functions, the bank distributed its various employees, numbering from 350 to 450 in the period 1974-78, largely of a clerical or managerial type, among 16 departments. The job ratings of its employees ranged from pay grade 3 to pay grade 16, with an ungraded officer group of about 8. New employees were generally assigned to pay grades 3 or 4 and assignments among departments were based on "educational background and prior work experience." All employees were evaluated annually on a scale of 1 (unsatisfactory) to 5 (exceptional). Since 1973, the branch had generally posted notices of vacancies by advertising in Southern Accent, a news circular prepared by and distributed to bank employees. Employees were invited to indicate their interest in any posted vacancy. Promotions were generally made within the work force.

In its complaint the EEOC charged the defendant with engaging in racially discriminatory practices and policies in failure to promote blacks at its Charlotte, North Carolina, branch in violation of Section 703(a) of Title VII, 42 U.S.C. Sec. 2000e. After the commencement of the action, four former or present employees of the defendant at the Charlotte branch petitioned to intervene in order to assert under Sec. 1981, 42 U.S.C., and Title VII individual and class claims of racial and sex discrimination "in promotions, wages, job assignments and terms and conditions of employment" on behalf "of all blacks and females who worked for the defendant at any time since July 2, 1965." The petition to intervene was allowed and the intervenors were, by a consent order, certified as the class representatives to maintain an action charging racial discriminatory practices and policies in the particulars stated in the petition for certification filed by the intervenors but with the class narrowed to include only employees who may have been hired after January 3, 1974. In the same consent order, the EEOC itself agreed to limit its claim of discrimination to "only ... those black persons who worked for the defendant since January 3, 1974."

After joinder of issues and considerable discovery, the actions both of the EEOC and of the plaintiffs-intervenors came on for trial in September, 1980. Following the completion of the trial, the District Court on October 29, 1980, filed its "Memorandum of Decision." It ruled in this Memorandum: (1) That the defendant had discriminated against the intervenor Cooper "by failing to promote her from her job as a settlement clerk ... to a position as utility supervisor" and against the intervenor Russell "by failing to promote her to a utility clerk position from her position as a utility operator" and "by discharging her ... in retaliation for her filing charges of discrimination with the Equal Employment Opportunity Commission;" (2) that the intervenors Moore and Hannah had not "shown the court that they suffered any discrimination on account of their race" and that their claims should be denied; and (3) that defendant had engaged in a pattern and practice of discrimination from 1974 through 1978 by failing to "afford black employees opportunities for advancement and assignment equal to opportunities afforded white employees [only] in pay grades 4 and 5." The "Memorandum" concluded with the direction to "[c]ounsel for plaintiffs ... to propose and submit by December 1, 1980: 1. Proposed findings of fact and conclusions of law consistent with the above findings ...."

The District Court filed on May 29, 1981, findings of fact and conclusions of law. In these it found discrimination by the defendant in the class action, in pay grades 4 and 5, and in individual discrimination claims of the intervenors, Russell and Cooper. It dismissed the individual discrimination claims of Hannah and Moore. While the District Court, as had the District Court in Amstar Corp. v. Domino's Pizza, Inc., 615 F.2d 252, 258 (5th Cir.), cert. denied, 449 U.S. 899, 101 S.Ct. 268, 66 L.Ed.2d 129 (1980), stated that "[t]he findings and conclusions herein, however, as well as the Judgment which follows are those of the Court based on an independent review of the record and consideration of the submissions of the parties," such statement was adopted verbatim from the plaintiffs' proposed findings and conclusions; in fact, the statement appears in exactly the same words and in the exact same place as footnote 3 in both the proposed findings and conclusions submitted by the plaintiffs and in the District Court's findings of fact and conclusions of law. Moreover, the Court's 37-page findings and conclusions were almost word for word copies of the findings and conclusions submitted by the plaintiffs.

From the judgment entered pursuant to the findings and conclusions of the District Court, as well as from an order granting an interim allowance of attorney's fees to the intervenors-plaintiffs' counsel, the defendant has appealed. The plaintiffs did not appeal the dismissal of the individual claims of the intervenors Hannah and Moore or the denial of relief in the class claim in all pay grades above pay grade 5. We reverse.

In considering such appeal we shall treat first the decision in the class action claim and, second, the decision on the individual claims of intervenors Cooper and Russell. Before addressing the substantive merits of the class action claim, however, it is necessary to resolve two preliminary points pressed by the defendant. The first of these relates to the nature or type of the class action claims, i.e., are they disparate treatment or disparate impact claims or both? The District Court's "Memorandum of Decision" does not identify the class action claim as either a disparate treatment or a disparate impact claim but in the findings and conclusions later adopted by the Court it is clear that the class action is being treated as both a disparate treatment and disparate impact claim. The defendant, on the other hand, asserts that the class action should be treated solely as a disparate treatment action. We agree.

In Stastny v. Southern Bell Tel. & Tel. Co., 628 F.2d 267 at 274, n. 10 (4th Cir.1980), we stated the necessary elements of a disparate impact claim. These elements, as there declared, are:

"As is now well recognized, the class action commonality criteria are, in general, more easily met when a disparate impact rather than a disparate treatment theory underlies a class claim. The disparate impact 'pattern or practice' is typically based upon an objective standard applied evenly and automatically to affected employees: an intelligence or aptitude test, e.g., Griggs v. Duke Power Co., 401 U.S. 424, 91 S.Ct. 849, 28 L.Ed.2d 158 (1971); an educational requirement, id.; a physical requirement, e.g., Weeks v. Southern Bell Tel. & Tel. Co., 408 F.2d 228 (5th Cir.1969). Both the existence and the 'common reach' of such objectively applied patterns or practices are likely to be indisputable from the outset, so that no real commonality problems for class action maintenance ever arise in this regard. On the other hand, the disparate treatment pattern or practice must be one based upon a specific intent to discriminate against an entire group, to treat it as a group less favorably simply because of its sex (or other impermissible reason). The greater intrinsic difficulty in establishing the existence and common reach of such a subjectively based practice is obvious. See Hauck v. Xerox Corp., 78 F.R.D. 375, 378 (E.D.Pa.1978). In the instant case, it is clear that plaintiffs' ultimate reliance would of necessity have been upon showing a pattern of disparate treatment. There is no suggestion in the record of a Griggs-type objectively imposed practice having discriminatory disparate impact."

We reiterated those criteria for a disparate impact claim in the recent case of Pope v. City of Hickory, N.C., 679 F.2d 20, 22 (4th Cir.1982):

" 'The disparate impact model applies only when an employer has instituted a specific procedure, usually a selection criterion for employment, [such as an aptitude or intelligence test, or height and weight requirements] that can be shown to have a causal connection to a class based imbalance in the [employer's] work force' and has been said not to be 'the appropriate vehicle from which to launch a wide ranging attack on the cumulative effect of a company's employment practices.' Pouncy v. Prudential Ins. Co. of America, 668 F.2d 795, 800 (5th Cir.1982). It is obvious that the plaintiff is not complaining in this case of some employment practice or procedure of the defendant, which, though neutral or fair on its face, has a discriminatory impact on blacks and thus does not fit within the model disparate impact claim."

It is manifest that the plaintiffs' class action claim does not meet the criteria for a disparate impact claim as those criteria are identified in Stastny and Pope. There was no evidence whatsoever of any "objective standard, applied evenly and automatically" in promotions, such as a physical requirement with respect to height, as in Dothard v. Rawlinson, 433 U.S. 321, 324, 97 S.Ct. 2720, 2724, 53 L.Ed.2d 786 (1977), or a high school diploma, as in Griggs v. Duke Power Co., 401 U.S. 424, 427, 91 S.Ct. 849, 851, 28 L.Ed.2d 158 (1971), or a minimum passing score on an aptitude test, as in Albemarle Paper Co. v. Moody, 422 U.S. 405, 410-11, 95 S.Ct. 2362, 2368, 45 L.Ed.2d 280 (1975) and Connecticut v. Teal, --- U.S. ----, 102 S.Ct. 2525, 73 L.Ed.2d 130 (1982). The claim here is a pattern or practice of intentional discrimination against an entire group by treating it less favorably because of race. That is the typical disparate treatment case. This case should accordingly be properly treated as such. However, the result reached by us would not be substantially different whether the class action be considered as a disparate impact or a disparate treatment case. Cf., Wright v. National Archives & Records Service, 609 F.2d 702 (4th Cir.1979).

Secondly, the defendant questions the weight, if any, to be accorded by us to the findings of fact and conclusions of law in this case. It is defendant's contention that the circumstance that these findings and conclusions were prepared by plaintiffs' counsel at the direction of the District Court and were adopted by the Court practically verbatim weakens, if it does not undermine completely, the reliability of and the weight to be accorded such findings and conclusions.2 There can be no dispute that the District Court itself, in its "Memorandum of Decision," actually made no findings of fact or conclusions of law as those terms are used and construed in Rule 52(a), Fed.R.Civ.P. It confined itself in this Memorandum to a purely conclusory statement that "the defendant [had] engaged in a pattern and practice of discrimination from 1974 through 1978 by failing to afford black employees opportunities ... afforded white employees in pay grades 4 and 5." Such a statement of ultimate fact is not a finding of fact reviewable under the "clearly erroneous" rule, and sustainable only if adequate supportive subsidiary findings are made. Hicks v. United States, 368 F.2d 626, 631 (4th Cir.1966); Castaneda v. Pickard, 648 F.2d 989, 1001 (5th Cir.1981) (with particular reference to a "discrimination" finding in Title VII actions). Apart from this statement, the District Court stated only that the "[d]efendant [had] not submitted statistical evidence rebutting plaintiff-intervenor's case with respect to discrimination in those grades." It did direct counsel for the plaintiffs to submit "Proposed findings of fact and conclusions of law consistent with the above [conclusory] findings" of discrimination and of non-rebuttal. It is the findings and conclusions so submitted by plaintiffs' counsel which the defendant attacks as entitled to little or no weight for accepting without question the plaintiffs' contentions as stated in the submitted findings, and for disregarding entirely in those findings the evidence and contentions offered by it.

We, along with other courts, have on a number of occasions--one as recently as a few months ago in Holsey v. Armour & Company, 683 F.2d 864 (4th Cir.1982)--expressed our disapproval of a trial court's practice of announcing its decision and then requesting the prevailing party to prepare findings of fact and conclusions of law which the court adopts almost word-for-word in support of its previously announced decision. The reason for such disapproval is inherent in Rule 52(a), Fed.R.Civ.P., a fair compliance with which "requires the trial court to find the facts on every material issue, including relevant subsidiary issues, and to 'state separately' its conclusions thereon with clarity." Kruger v. Purcell, 300 F.2d 830, 831 (3d Cir.1962); De Medina v. Reinhardt, 686 F.2d 997, 1011 (D.C.Cir.1982).3 As the Court in Sims v. Greene, 161 F.2d 87, 89 (3d Cir.1947), said in language quoted and approved by us in Consolidation Coal Co. v. Disabled Miners of So.W.Va., 442 F.2d 1261, 1269 (4th Cir.), cert. denied, 404 U.S. 911, 92 S.Ct. 228, 30 L.Ed.2d 184 (1971), "[t]he conclusion is inescapable that since a district court is required by the rule [Rule 52(a) ] to make findings of fact, the findings must be based on something more than a one-sided presentation of the evidence," or, as the Court in the same opinion repeated, "[f]inding facts [under Rule 52(a) ] requires the exercise by an impartial tribunal of its function of weighing and appraising evidence offered, not by one party to the controversy alone, but by both."4 See to the same effect: McManus v. Midland Valley Lumber Co., 348 F.2d 898, 900 (4th Cir.1962) ("... must necessarily consider all available evidence bearing upon the issue"); Burgess v. Farrell Lines, Inc., 335 F.2d 885, 889 (4th Cir.1964); Sligh v. Columbia, Newberry and Laurens Railroad Co., 250 F.Supp. 490, 491 (D.S.C.1966), aff'd. 370 F.2d 979 (4th Cir.), cert. denied, 386 U.S. 1007, 87 S.Ct. 1349, 18 L.Ed.2d 434.

This application of Rule 52(a), it is true, does not require the trial court to deal with every piece of evidence in the record or every argument made during the proceedings, whatever their value, but it does mean that "[t]he reviewing court deserves the assurance [given by even-handed consideration of the evidence of both parties] that the trial court has come to grips with apparently irreconcilable conflicts in the evidence ... and has distilled therefrom true facts in the crucible of his conscience." Golf City, Inc. v. Sporting Goods Co., Inc., 555 F.2d 426, 435 (5th Cir.1977).

All these considerations prompted the Supreme Court in U.S. v. Crescent Amusement Co., 323 U.S. 173, 184-85, 65 S.Ct. 254, 259-260, 89 L.Ed. 160 (1944) to comment that the adoption of "findings [proposed by one of the parties to the suit and adopted by the trial judge] leave much to be desired in light of this function of the trial court," under the Rules. This is so because an appellate court will " 'feel slightly more confident in concluding that important evidence has been overlooked or inadequately considered' when factual findings were not the product of personal analysis and determination by the trial judge." James v. Stockham Valves & Fittings Co., 559 F.2d 310, 314, n. 1 (5th Cir.), cert. denied, 434 U.S. 1034, 98 S.Ct. 767, 54 L.Ed.2d 781 (1978). Nor is that disquiet, prompted by the trial court's adoption of one party's findings and conclusions, relieved by any statement in such findings and conclusions that the trial court had " 'individually considered' them and adopted them because it 'believed them to be factually and legally correct'; [even though] a cursory reading of the district court's memorandum leaves one with the impression that it was indeed written by the prevailing party to a bitter dispute." Amstar Corp. v. Domino's Pizza, Inc., supra, 615 F.2d at 258.

The adoption by the District Court of proposed findings and conclusions, though disapproved, will not, however, warrant reversal of the cause per se nor does it mean that the " 'clearly erroneous' " rule of Rule 52(a) will not be applied at all, simply because the findings and conclusions were developed by one of the parties and adopted in course by the judge. As the Court in Flowers v. Crouch-Walker Corp., 552 F.2d 1277, 1284 (7th Cir.1977), after observing that "the district court [had] adopted [in that case] without change findings of fact and conclusions of law prepared by the defendant," said: "[a] critical view of a challenged finding is appropriate where, as here, the findings of fact and conclusions of law of which it is a part were not the original product of a disinterested mind." Again, in Photo Electronics Corp. v. England, 581 F.2d 772, 777 (9th Cir.1978), the Ninth Circuit expressed itself similarly, declaring that, while "the fact that the trial judge has adopted proposed findings does not, by itself, warrant reversal ... it does raise the possibility that there was insufficient independent evaluation of the evidence and may cause the losing party to believe that his position has not been given the consideration it deserves. These concerns have caused us to call for more careful scrutiny of adopted findings." See also, United States v. State of Wash., 641 F.2d 1368, 1371 (9th Cir.1981), cert. denied, 454 U.S. 1143, 102 S.Ct. 1001, 71 L.Ed.2d 294, ("Verbatim adoption of proposed findings of fact by the district court ... calls for close scrutiny by an appellate court"); and Shlensky v. Dorsey, 574 F.2d 131, 149 (3d Cir.1978) (such adoption requires the appellate court to "examine them more narrowly").

When the findings of fact and conclusions of law adopted by the District Court have been given that "careful scrutiny" by the appellate court that is required under such circumstances and have been "more narrowly" examined than findings and conclusions which, because developed independently by the trial judge, provide assurance that the District Judge making the findings and conclusions "did indeed consider all the factual questions thoroughly and ... guarantee[s] that each word in the finding [was] impartially chosen," Louis Dreyfus & Cie. v. Panama Canal Co., 298 F.2d 733, 738 (5th Cir.1962, Wisdom, J.), and when, the reviewing court, on the entire record, "is left [after such review] with the definite and firm conviction that a mistake has been committed," United States v. Gypsum Co., 333 U.S. 364, 395, 68 S.Ct. 525, 541, 92 L.Ed. 746 (1948), or it is convinced that "the result in a particular case does not reflect the truth and right of the case," Armstrong Cork Co. v. World Carpets, Inc., 597 F.2d 496, 501 (5th Cir.1979), cert. denied, 444 U.S. 932, 100 S.Ct. 277, 62 L.Ed.2d 190, it is the duty of the appellate court to reverse the findings and conclusions as clearly erroneous.

It is important, too, to note, before addressing the merits of the case, the limited nature of the District Court's finding of discrimination in the class action, as declared by it in both its "Memorandum of Decision" and in the subsequent findings of fact and conclusions of law adopted by it. In its "Memorandum," the District Court, while finding discrimination in affording "black employees opportunities for advancement and assignment equal to opportunities afforded white employees in pay grades 4 and 5," made it clear that "[o]ther than in the above particulars, [i.e., in promotions in pay grades 4 and 5], however, there does not appear to be a pattern and practice of discrimination pervasive enough for the court to order relief."

This narrowing of the issues in the class action to promotions out of pay grades 4 and 5 was restated in the later findings and conclusions. Thus, finding of fact # 56, as adopted by the District Court is:

"Except for promotions from pay grades 4 and 5, plaintiffs' and defendant's data ... indicated no statistically significant difference in the initial job assignments and pay grades, performance evaluations or promotion of black and white employees."

The same ruling was included in the District Court's conclusions of law # 27:

"The Court concludes that there was no showing that the bank had discriminated against black employees with respect to promotions out of grades 6 and above, and that defendant did not violate Title VII or 42 U.S.C. Sec. 1981 with respect to promotions out of grade 6 and above."

Moreover, the finding of discrimination in pay grades 4 and 5 was one of a pattern and practice of discrimination in those grades. The establishment of a pattern or practice of discrimination requires proof of "more than the mere occurrence of isolated or 'accidental' or sporadic discriminatory acts" and depends upon a finding by "a preponderance of the evidence that racial discrimination was the [defendant's] standard operating procedure--the regular rather than the unusual practice." Teamsters v. United States, 431 U.S. 324, 336, 97 S.Ct. 1843, 1854, 52 L.Ed.2d 396 (1977).

With the issues in the class action claim thus limited the District Court proceeded to state its conclusions on those issues (Conclusion # 17):

"Black employees assigned to pay grades 4 and 5 have, during the relevant time period, been retained in these grades for longer periods than comparable white employees and passed over for promotion solely because of their race and color. Using two statistical methods, plaintiffs demonstrated disparate treatment by defendant of black employees in these grades, that black employees have been retained in these grades for significantly longer periods even when their relative qualifications are equal to white employees and that the disparate treatment is statistically significant at the 5 percent level."

The Court earlier in its findings had set forth the factual basis for that conclusion. It declared in such findings that the statistical evidence proffered by the plaintiffs and the defendant as well as "[t]he oral testimony of class members" had established "a prima facie case that black employees in pay grades 4 and 5 [had] been denied promotions from these grades solely because of their race," that, in response, the defendant had "offered no explanation for its unfavorable treatment of black employees in pay grades 4 and 5," and "therefore [it found], based on all of the evidence of record, that black employees who [had] been assigned to pay grades 4 and 5 between 1974 and the date of trial [had] been deprived of rights under Title VII." It is manifest from these statements that the District Court's findings and conclusions rest on "the statistical evidence" in the record and "the oral testimony of class members."Before examining the statistical evidence relied on by the District Court for support for these findings and conclusions, we would review the "oral testimony of class members" which the District Court found supplemented the statistical evidence in establishing discrimination in pay grades 4 and 5. There were only three "class members" who testified, and to whom the finding of supportive "oral testimony of class members" could apply in connection with discrimination in promotions out of pay grades 4 and 5. These were Alfred Harrison, Elmore Hannah and Emma Ruffin, they being the only live employee-witnesses who were members of a class of employees not promoted out of pay grades 4 and 5.5 The claim of Elmore Hannah, one of the three non-promoted witnesses in pay grades 4 or 5, was fully discussed and the District Court made very clear findings on his claim. The Court found that Hannah had been "employed through the Bank's handicapped program" at pay grade 3 and, when tried, at higher grade level jobs, had "failed to demonstrate that he was qualified or able to perform the job positions he requested." His claim was accordingly dismissed by the District Court as without merit. The claim of Hannah was accordingly unavailable as a support for any finding of discrimination; indeed, the defendant's efforts at placing Hannah in higher level jobs demonstrated, if anything, an absence of discriminatory practice on defendant's part against handicapped blacks. That leaves the "oral testimony" of Ruffin and Harrison6 as that of the only "class members," out of the countless numbers who had been employed in these two classes, who could be said to support the statistical evidence on which the District Court rested its findings of a pattern of discrimination, as found in this case.

This case accordingly presents quite a contrast with Teamsters where the "oral testimony of class members" demonstrated 40 cases of specific instances of discrimination in support of the statistical evidence offered by plaintiffs or with that in our own case of Chisholm v. United States Postal Service, 665 F.2d 482, 495 (4th Cir.1981), where there were 20 "class members" testifying of individual discrimination. Here all we have is the testimony of but two class members testifying of individual discrimination in promotion out of either pay grade 4 or pay grade 5 on which a finding of discriminatory practices can be rested. This is even less of a presentation of oral testimony in support of a pattern of discrimination than that found wanting in Ste. Marie v. Eastern R. Ass'n, 650 F.2d 395, 405-06 (2d Cir.1981), where the Court declared that the small number of incidents of discrimination in promotion over a period of years in that case "would be insufficient to support the inference of a routine or regular practice of discrimination ...,"7 or, in Goff v. Continental Oil Co., 678 F.2d 593, 597 (5th Cir.1982), where the Court held that "even if all three witnesses' accounts of racial discrimination were true, this evidence would not have been enough to prove a pattern or practice of company-wide discrimination by Conoco." It follows that these two incidents of failure to promote Ruffin or Harrison, even if regarded as discriminatory, (which we assume only arguendo ), would not support the District Court's finding of a pattern of class discrimination in promotions out of grades 4 or 5 or offer any reinforcement to an inference of discrimination derived from statistical proof; under Ste. Marie v. Eastern R. Ass'n and Goff, there was an absence of sufficient evidence, based on class members' testimony, to provide a basis for a finding of a pattern of discrimination in promotion out of pay grades 4 and 5.

It is true that a number of "live" witnesses other than Ruffin and Harrison testified for the plaintiffs. This number included the four plaintiffs Cooper, Moore, Hannah and Russell and certain other past or present employees who moved, after adverse decision, to intervene as plaintiff-intervenors. This latter group consisted of Phyllis Baxter, Brenda Gilliam, Glenda Knott, Alfred Harrison and Sherri McCorkle. In denying the motion the District Court stated that all intervenors "in grades higher than grade 5" were not members of the class in whose favor the District Court had found "classwide discrimination." By this test, Cooper, Moore, Russell, Baxter, Gilliam, Knott and McCorkle were not members of the class in which discrimination was found and their testimony could not have been included within the District Court's term "oral testimony of class members," complaining of promotion out of either pay grade 4 or pay grade 5; only the testimony of Ruffin and Harrison met that qualifying standard. However, it is interesting to review the employment records of these witnesses, other than Ruffin and Harrison, in order to see how they were promoted out of pay grades 4 and 5. Thus Baxter hired at pay grade 5, was promoted to pay grade 6 seven months after she was hired. Gilliam, hired at pay grade 4, was promoted to pay grade 5 within approximately 17 months, and within eleven months afterward to pay grade 6; McCorkle, hired at pay grade 4, had in 24 months been promoted two grades to grade 6; and Knott, hired at grade 5, was promoted to grade 6 within 6 months after employment at the Charlotte branch. Russell was employed at pay grade 4, was promoted twelve months later to pay grade 5, and within a month or so, to pay grade 6. Cooper was employed initially at pay grade 3, was promoted two months later to pay grade 4, fifteen months later to pay grade 5, eleven months later to pay grade 6. The intervenor Hannah was found by the District Court not to have suffered discrimination.

The experiences of these seven black employees, (Cooper, Russell, Moore, Baxter, Gilliam, Knott and McCorkle) all of whom, with the exception of Ruffin, Harrison and Hannah, had promoted out of pay grades 4 and 5 in periods either less than or equivalent to the average of white employees, far from supporting any proof of discrimination in promotions out of pay grades 4 and 5, represented strong proof of the absence of class-wide discrimination in promotion out of pay grades 4 and 5; and since Ruffin's and Harrison's testimony, which is the only "oral testimony of class members" that could in any circumstances be said to support a charge of a pattern of class discrimination, is insufficient, the District Court's finding of a pattern of class discrimination can find no support in the "oral testimony of class members" in pay grades 4 and 5 and must find its basis in the statistical evidence and in that evidence alone.8

Statistics, when properly authenticated, constitute an accepted form of circumstantial evidence of discrimination and may sometimes be sufficient to establish without more prima facie proof of discrimination. But statistics "come in infinite variety" and their usefulness or weight "depend[s] on all of the surrounding facts and circumstances," Teamsters v. United States, 431 U.S. at 340, 97 S.Ct. at 1856, and on " 'the existence of proper supportive facts and the absence of variables which would undermine the reasonableness of the inference of discrimination which is drawn' " therefrom, White v. City of San Diego, 605 F.2d 455, 460 (9th Cir.1979). Inaccuracies or variations in data or in the formulae used to test such data may easily lead to different, contradictory, or even misleading conclusions by experts. This fact prompted one court to comment that too often statistical conclusions "appear to depend in large part on the side producing them ...." Stastny v. Southern Bell Tel. & Tel. Co., 458 F.Supp. 314, 324 (W.D.N.C.) aff'd. in part and rev'd. in part, 628 F.2d 267 (4th Cir.1980). And the sophisticated way in which supporting data may be used in developing statistical models in discrimination cases has lead another Court to caution about "the manipulability of statistics in inquiries of [that] sort," Bilingual Bicultural Coalition, Etc. v. F.C.C., 595 F.2d 621, 625, n. 7 (D.C.Cir.1978), and still another to suggest that Title VII cases too often develop into "contests between college professor statisticians who revel in discoursing about advanced statistical theory" and propounding increasingly complex statistical models. Otero v. Mesa Cty. Valley Sch. Dist. No. 51, 470 F.Supp. 326, 331 (D.Colo.), aff'd., 628 F.2d 1271 (10th Cir.1980).9

We do not mean to suggest that statistical conclusions, supported by adequate and accurate supporting data and developed through the use of neutral and impartial tests, are not to be given weight, sometimes compelling weight if there is no rebutting evidence, in resolving claims of racial and sex discrimination. We have repeatedly relied on such evidence in a proper case in reaching our decisions and the Supreme Court itself has approved the use of such evidence. In fact, as we said in Equal Employment Opportunity Com'n v. Am. Nat. Bank, 652 F.2d 1176 (4th Cir.1981), a "prima facie showing may in a proper case be made out by statistics alone (citing Teamsters,10 Hazelwood,11 and Barnett12 ), or by a cumulation of evidence, including statistics, patterns, practices, general policies, or specific instances of discrimination." Id. at 1188. But statistical evidence, like any other type of circumstantial evidence, "must not be accepted uncritically," Logan v. General Fireproofing Company, 521 F.2d 881, 883 (4th Cir.1971), and, because of the sophistication and complexity of many of the statistical models being used in discrimination cases by professional econometricians, courts must give "close scrutiny [to the] empirical proof" on which the models are erected, Pettway v. American Cast Iron Company, 494 F.2d 211, 231, n. 44 (5th Cir.1974), in order to guard against the use of statistical data which may have been "segmented and particularized and fashioned to obtain a desired result," Equal Employment Opportunity v. Data Point Corp., 570 F.2d 1264, 1269 (5th Cir.1978). As one authority in the field of discrimination litigation has stated: "[T]he Supreme Court's directives in Teamsters and Hazelwood to evaluate statistical proofs clearly in light of all relevant circumstances reinforce decisions like Robinson v. Dallas,13 Olson v. Philco-Ford14 and Keyes v. Lenoir-Rhyne15 and make clear that in no case should there be a blind adherence to the proposition that mere statistical imbalance equals discrimination." Morris, Current Trends in the Use (and Misuse) of Statistics in Employment Discrimination Litigation, Second Edition, 1979, Equal Employment Advisory Council, p. 51.16 "To be legally cognizable, the pattern [of disparity] revealed must be at least 'significantly discriminatory,' Dothard v. Rawlinson, 433 U.S. 321, 97 S.Ct. 2720, 2727, 53 L.Ed.2d 786 (1977); at minimum, the percentages must be 'markedly disproportionate,' Griggs v. Duke Power Company, 401 U.S. 424, 91 S.Ct. 849, 852, 28 L.Ed.2d 158 (1971);" and "[s]tatistical proof failing to show a 'marked disproportion', Griggs, 91 S.Ct. at 852, by definition cannot show the 'gross disparity,' Teamsters [97 S.Ct.] at 1856, n. 20, necessary to sustain allegations of disparate treatment." Rivera v. City of Wichita Falls, 665 F.2d 531, 534-35 and 535, n. 5 (5th Cir.1982).

Of course, statistical evidence, like any other evidence, is always subject to rebuttal and this rebuttal may assume a number of forms. In Dothard v. Rawlinson, Justice Rehnquist, concurring, said that the defendants in a discrimination case "may endeavor [in rebuttal] to impeach the reliability of the statistical evidence, they may offer rebutting evidence, or they may disparage in arguments or in briefs the probative weight which the plaintiffs' evidence should be accorded." [433 U.S. at 338-39, 97 S.Ct. at 2731] And in Teamsters, the Supreme Court declared that statistical evidence may be rebutted by "demonstrating that [the plaintiff's] proof is either inaccurate or insignificant." [431 U.S. at 360, 97 S.Ct. at 1867.] A similar observation was made by us in Roman v. ESB, Inc., 550 F.2d 1343, 1350 (4th Cir.1976):

"We do not believe that isolated bits of statistical information necessarily make a prima facie case when divorced from other and contrary statistics and from the statistical picture of all the employment at the plant. We also think the absence of other evidence of discrimination should be considered in determining whether a prima facie case is made, just as the presence of other evidence of discrimination should be considered in arriving at the same conclusion."

To sum up, statistical evidence is circumstantial in character and its acceptability depends on the magnitude of the disparity it reflects, the relevance of its supporting data, and other circumstances in the case supportive of or in rebuttal of a hypothesis of discrimination. And, in reviewing statistical evidence and its supporting data, the Court must give consideration and evaluate fairly such conflicting opinions and hypotheses as may have been presented, tempering its conclusion with what one Court has described as "a pinch of common sense." Otero v. Mesa Cty. Valley Sch. Dist. No. 51, [470 F.Supp. at 335].

We should also note an important issue that arises in any review of statistical evidence and this is the determination of the meaning of the term statistical significance in this context.17 There are numerous rules stated by econometricians for determining "statistical significance" in discrimination cases, though as Agid, Fair Employment Litigation, at 541 puts it, "[t]here are no hard and fast rules as to how much of a disparity, is 'enough' to establish a prima facie case or withstand various defenses." Some statisticians base their opinion on the "five per cent level" of disparity in black and white employment as the measuring standard for statistical significance in discrimination cases. A few even in some circumstances, using "a one-tailed probability level to facilitate obtaining 'significant' results," employ a level of 1.64 standard deviations for their opinion of statistical significance. See Friedman, Introduction to Statistics, 146 (Random House, 1972). However, "[t]he adoption of a particular level or test of statistical significance, ... is arbitrary," Smith and Abram, Quantitative Analysis and Proof of Employment Discrimination, 1981 U.Ill.L.Rev. 33 at 43, and a recent commentator has wisely cautioned that "modern statisticians are critical of using five percent or any other level as an absolute standard of significance" in this connection.18

The Supreme Court itself, though disclaiming any intention "to suggest that precise calculations of statistical significance are necessary in employing statistical proof," has stated that standard deviations of more than "two or three" represent a minimum for statistical significance.19 Obedient to our understanding of this rule of the Supreme Court in Hazelwood School District v. United States, 433 U.S. at 311-12, n. 17, 97 S.Ct. at 2743, n. 17, and Castaneda v. Partida, 430 U.S. 482, 97 S.Ct. 1272, 51 L.Ed.2d 498 (1977), we have adopted the rule that the proper method for determining "legal significance" on the basis of statistical evidence is through the use of the standard deviation analysis, and in the cases where either the Supreme Court or we have used that standard, we have followed the binomial distribution test. Moultrie v. Martin, 690 F.2d 1078 (4th Cir.1982). Moreover, in American National Bank, supra, we held in interpreting the decisions in Hazelwood and Castaneda, as we understood them, that courts "should be extremely cautious in drawing any conclusions [of legal significance] from standard deviations in the range of one to three," but that a statistical analysis "with standard deviations of more than three" could "safely be used ... absolutely to confirm" an inference of some "disparity."20 It will be noted that we have used the term "legal significance" in this context, rather than "statistical significance." We have done so advisedly for in this connection "statistical significance" to the econometrician may not be and often is not the same as "legal significance," the determination of which, after all, is an issue solely for the court and not for an expert witness.21 This is recognized in Baldus and Cole, Statistical Proof of Discrimination, 308 (McGraw-Hill, 1980) in which the authors state that "what is or is not statistically significant, that judgment is a legal determination properly made by the court and not by an expert." Thus, we emphasize that number comparisons which have statistical significance may not necessarily have legal significance but, while statistical significance as measured by the standards of acceptable statistical principles will not necessarily be legally significant, a finding of legally significant variations based on statistical evidence may not be made in the absence of a finding of statistical significance within acceptable confidence levels in any event.

With these principles in mind, we approach the statistical evidence on which the District Court in this case relied for the result it reached. The centerpiece in the statistical findings made by the District Court is represented in its Finding # 57, which basically consists of two Tables. It was primarily on the basis of these two Tables, as submitted in the testimony of plaintiffs' expert, that the District Court found "statistically significant disparity" in promotions of blacks in pay grades 4 and 5 in this case. These Tables are an exact reproduction of exhibits 34a and 35a (except that the tables in the finding omit the standard error figures as they appear on the exhibits) as introduced by the plaintiffs through their expert witness Dr. Hoffman. These two exhibits purport to show (1) the exact number of employees in the two relevant pay grades of 4 and 5 for the four years in question, (2) the percentage of blacks among such employees, (3) the total promotions in each year during such period, (4) the number of such promotees who were black, (5) the expected number of black promotees if black promotions had coincided precisely with the black percentage of the overall employee force in the two grades, and finally, (6) the difference between the expected number of black promotions and the actual number of blacks promoted in such pay grades in the relevant years.

There are a number of significant facts to be observed about the numbers used in these two Tables which are crucial in any examination of the District Court's findings derived therefrom. First, the number of employees in each pay grade for any one of the four years in issue, as listed in these Tables, is neither "Total Incumbents at Beginning of Interval" (that is, at the beginning of the year in question) nor, as the Tables themselves state, "Total Incumbents at End of Interval" (that is, after the end of the year in question). The number of employees in the pay grade for any year, as used in the Tables, is the number of employees who were employed in the pay grade at the beginning of that year and who remained in the pay grade at the end of that year. In other words, any employee who quit, was promoted out, or was fired, or was replaced during the year is not counted in this calculation. This method of calculating the employee numbers out of which promotions were to be made in any stated year increased, particularly in pay grade 4, the percentage of black employees over what it would have been had the Tables used the actual number of employees in the pay grade at the beginning of the stated year, or the actual number at the end of the year, or any average of the two.

More important for meaningful analysis than this method of calculating the employee mass for determination of the sample is the manner in which the expert calculated the number of black promotions made in the grades for each of the years in question. It was undisputed that the actual number of black promotions in pay grade 4 for the years 1974-1977, for instance, was 39 but the Tables used by the District Court gave this number as 35. The reason for the difference is that any employee, whether black or white, who, after promotion might have terminated voluntarily or involuntarily, was simply eliminated from the calculations of black promotees during that year, as set forth in these Tables. Plaintiffs' witness admitted that such a procedure where only the number of black promotees who, after promotions, continued to be employed and not the actual, correct number of black promotions was used in the calculations, "change[d] the statistics dramatically in the case of grade 4."22

The reasons assigned by the expert witness for the use of such artificial numbers on promotions of blacks during the relevant years in his Tables were that, to quote the expert, "there may be promotions that are given to individuals which are not permanent in nature, which leave, which the incumbent leaves quickly after he is promoted and for a variety of reasons--either he's going back to school or he's dissatisfied with his work or he's dissatisfied with the promotions or he fails at the promotion and wishes to leave employment." All of these promotees should be eliminated in the calculations of annual black promotions in the expert's judgment. The District Court did not inquire into the reasonableness of this justification for the omission of such promotees in stating the number of black promotions for a fair and impartial analysis in the critical years, nor are we able to find any basis in the record for such a justification. It seems difficult to assume that when an employee, whether white or black, has requested a promotion to a particular job vacancy (that is the way the evidence shows promotions were generally made) he would be "dissatisfied with the promotion" he had sought and would quit because he got it. Similarly, it is a little odd that, when it is the plaintiffs' contention (which, incidentally, is somewhat specious, as we see later) that it took an average of almost four years for a black to be promoted out of grade 4, the defendant would be promoting a black schoolboy, whose work life would normally be no more than the three school vacation months and who would quit at the end of his school vacation. Moreover, if the question is whether the defendant intentionally failed to promote blacks in pay grades 4 and 5 in a particular year, it would seem that the correct test figure should be the actual promotions made in that particular year out of those pay grades. We are unable to perceive any rational basis for using an inaccurate figure for promotions during the pertinent years for black promotions in exhibits 34a and 35a unless it was "to obtain a desired result" of a standard deviation in excess of -2. Cf., Equal Employment Opportunity v. Datapoint Corp., 570 F.2d 1264 (5th Cir.). Certainly, by reducing the number of promotions, the expert increased the standard deviation but it was an increase achieved not by analyzing the actual numbers but by reducing the number of the actual promotions, thereby diminishing significantly the validity of any calculation of standard deviation based on such artificial numbers. We find exhibits 34a and 35a are fatally flawed by the manner in which they were prepared and by the assumptions on which they were based.

It is, however, equally interesting to see the method which was adopted without question by the District Court in its critical finding of standard deviations as shown by these Tables. The District Court, in these findings, accepted without question the plaintiffs' expert's calculation of standard deviations in both grades 4 and 5 as shown in exhibits 34a and 35a. There are, however, two tests used in calculating standard deviation in a case such as this. The first and the one used by the Supreme Court in Castaneda and Hazelwood and by us in American National Bank, is the binomial distribution formula; the other is the hypergeometric distribution formula. Some statisticians indicate that the latter test may be used when small numbers are involved and when these numbers are "finite ... without replacements." Winkler and Hays, Statistics: Probability, Inference, and Decision, 225 (2d Ed. 1975; Holtetc); Hoel and Jessen, Basic Statistics for Business and Economics, 132-33 (2d Ed., 1977; Wiley); Hoel, Introduction to Mathematical Statistics, 67-68 (1971; Wiley). Baldus and Cole, in their 1982 Supplement suggest at p. 82, on the other hand, that the binomial test is proper when the sample is "at least 30" or more.23 The samples in exhibits 34a and 35a which incidentally use for the calculations the tables for all four of the relevant years combined and not the tables for one year, are substantial in number and more than meet these minimum number requirements. In pay grade 4 the sample number is 154 and in pay grade 5 it is 269. The size of the sample in either case warrants the use of the binomial test. Moreover, any terminations of employees during the period presumably were replaced and thus the numbers in the sample were not "finite without replacements." It would appear, therefore, that neither of the two reasons generally given for preferring hypergeometric distribution test over the binomial existed in this case. The two tests did, though, result in different standard deviations and different standard deviations more favorable to the plaintiffs. The hypergeometric test thus gave larger standard deviation results. This fact is demonstrated by the Table appearing in note 24 below,24 which is based on the same numbers as are used in exhibits 34a and 35a, comparing specifically the results under both a hypergeometric distribution test and a binomial test (a test, incidentally, the expert did not employ).

It will be observed that in computing the standard deviations in pay grade 5 even under the hypergeometric test, we reach a different result from that which the plaintiffs' exhibit 35a showed. Our Table shows a standard deviation of -1.87 and that of the plaintiffs' expert, accepted by and set forth by the District Court in its Finding of Fact # 57, is stated as -2.01. The reason for the difference is the difference in what the exhibit (as included in the Appendix at pages 1115-1117 but omitted in the District Court's findings) describes as "standard error." In the expert's Table, this figure is 3.73 and in the one used by us is 4.02. The basis on which we arrived at our result is shown in note 25.25 If our computation is correct, the standard deviation for pay grade 5 was -1.87, which, under the standard null hypothesis, as stated by plaintiffs' expert, would not be statistically significant. Further, if we compute the standard deviation under the binomial test for pay grade 4, the standard deviations for grade 4 in exhibit 34a would be -2.07 and for pay grade 5 in exhibit 35a would be -1.45. Neither standard deviation, as derived under the binomial test, would be statistically significant, since for pay grade 4 the deviations are but marginally over -2, and for pay grade 5 well below -2. Moreover, there are, as we shall see later, special circumstances which render the standard deviation in pay grade 4, even calculated as the plaintiffs' expert did under the hypergeometric test as -2.69, to be without legal significance.

NOTE: OPINION CONTAINS TABLE OR OTHER DATA THAT IS NOT VIEWABLE

NOTE: OPINION CONTAINS TABLE OR OTHER DATA THAT IS NOT VIEWABLE

NOTE: OPINION CONTAINS TABLE OR OTHER DATA THAT IS NOT VIEWABLEThe District Court cited and relied on another Table prepared by plaintiffs' expert. It was described by the expert as a "buddy" study in which, as he explained it, he sought to match black and white employees, with similar lengths of service, similar grades, similar educational levels, similar department and similar lengths of service in grades. The number of matches covered but a fraction of the defendant's workforce, and their number was not confined to employees in pay grades 4 and 5 but embraced employees selected from all levels of defendant's workforce. If we accept the expert's opinion that the study indicated discrimination in promotions between white and black employees, then the study would have indicated discrimination at all levels of employment in the Bank and would actually contradict the District Court's express finding that discrimination in promotions was confined to pay grades 4 and 5. It is unnecessary, however, to examine the reliability of this aspect of the study, since it was conceded by the expert that such study showed a finding of -1.79 standard deviations, which he said was ".037," evidence of disparity between black and white employees in promotions, or well below the standard of "between two to three," or even -1.95.

It was obvious at this point in the expert's testimony that he had given no firm opinion on this aspect of his study which would support a conclusion under the econometrician's standards of statistical significance in promotions out of pay grades 4 and 5. The plaintiffs' counsel apparently recognized this and, in seeking to overcome this obvious deficiency in proof, requested the expert to explain at this point "what does this (i.e., the buddy study) tell us about '74 through '77 with your buddy system." To that question the expert replied, "[t]hat if you had reason to believe that blacks were being treated differently than whites, that there was a significant difference." Apparently finding the answer ambiguous, counsel followed that answer with this question to his own expert:

"Q Must we assume that there was a difference in order to reach a conclusion?

"A If you assume that there was no difference and that blacks at this point in all of the studies that I've done were equally likely to be above or rated more highly than whites and promoted at a faster rate than whites as they were to be promoted at a slower rate than whites, you would have a 'P' value that was approximately twice as large as that, or .07, or 7 times in 100. Using the standard on a two-tailed test of 5 percent you would not have a significant difference."

When the expert gave this explanation of his results, the District Judge interrupted to inquire, "I say where in this data is it possible statistically to generate a hypothesis [or 'presumption'] which will beef up the actual results of the statistical analysis? .... where then does that hypothesis come from that race played a part?" (Italics added) The expert's answer was:

"A The two-tailed test or the test that assumes that blacks are as equally likely to be higher as they are to be lower, which is the alternative. In statistics you say, I believe that there is no difference. You then look to see if there is some relationship in this case between race and promotion. If you truly have no reason to suspect that blacks would be higher or lower, you must use the two-tailed test, but in an experimental design, for instance, where there is a substantial amount of information gathered prior to performing the study that indicates that there is a relationship between race or between what you're studying and an outcome--

"COURT: Are you saying that if you draw the conclusion by inspection that the figures are skewed on a racial basis then your statistical opinion depends of whether you put the question in the positive or negative?

"A Yes, and if you assume that the processes that are gone through in order to determine whether or not discrimination has occurred in order to get a right to sue letter or in the initial evaluation that there is a reasonable chance that it could have occurred in this spot is a proper assumption, you have the ability to use a stronger and a more powerful alternative hypothesis."

After hearing the expert's explanation, the District Judge inquired: "You weren't stating the hypothesis that is the speculative proposition that you're going to test against the figures you've got. You're talking in terms of a presumption or a reason to start out favoring one side or the other of the question." (Italics added) To that inquiry the expert responded: "It is a presumption, yes." Under these admissions of the expert, a finding of statistical significance, of a magnitude sufficient to support, even for the econometrician, an inference of discrimination could only be arrived at if one begins his review of the statistical evidence with a "stronger and more powerful alternative hypothesis" or "presumption" and one that admittedly favors the plaintiff in a discrimination case that there is "a reasonable chance" that there has been discrimination. Such an assumption, which arbitrarily favors one party to the controversy, cannot be considered a reliable basis for a finding of discrimination.

Moreover, any finding of discrimination in this case, based on the statistical evidence is compellingly rebutted by two other Tables prepared by the expert before he produced the Tables in exhibits 34a and 35a. These Tables were listed in the record as exhibits 34 and 35 and covered employees in the pertinent pay grades 4 and 5 for the same four years as exhibits 34a and 35a. Such Tables began by taking as the employee pool in the two relevant pay grades for analysis the "Total Incumbents [in those pay grades] at Beginning of Interval." Unlike the numbers in exhibits 34a and 35a, this is an accurate and exact number, and similarly, in setting forth the "Total Black Promotion" in any given year within the relevant time period for both pay grades the actual number of black promotions made during the year was used. In short, these Tables deal with actual, not artificial or tailored numbers and they present a precise picture of the percentage of black promotions in the relevant years for pay grades 4 and 5. The results stated in terms of standard deviations, as measured by the binomial test, are significantly different than those in the exhibits relied on by the District Court, i.e., -1.52 for pay grade 4 and -1.24 for pay grade 5, as shown by the compilation appearing in note 26.26

It will be observed that under Tables 34 and 35 the standard deviations, computed under the binomial test, are such that "the hypothesis that the [selection process] (for black promotions in pay grades 4 and 5) ... would be suspect to a social scientist," is not proved.27 The expert, however, did not use the binomial test in his Tables; he used again the hypergeometric test. The expert conceded that the standard deviation difference between whites and blacks in promotions out of pay grade 4 for the pertinent years under the hypergeometric test, as shown by these tables, was -1.89. On the statistical significance of this result, he testified:

"A 1.89 standard deviation difference between blacks and whites. This is a significant difference if one makes the assumption that it's reasonable to believe that blacks have been discriminated against. The value of that standard error given that assumption is 1.65 standard errors. That corresponds to 5 times in 100. If one makes the assumption that I don't know, that I can't tell and I have no reason to believe whether blacks were discriminated against or not, that is not a significant difference at the .05 level, which I believe the standard error would be 1.95.

"Q So for the year 1974 does this table reflect that blacks have not promoted out of grade 4 on a comparable basis with whites?

"A Not statistically."

As the Court remarked, after hearing this testimony:

"... to give any meaning to the conclusion expressed in Table 34 [sec. 4] you've got to have pretty well decided the case before you read Table [sec. 4] 34?

"That's true."

The expert did testify that exhibit 35 showed a significant statistical disparity in grade 5 under a hypergeometric test, (i.e., -2.24).28 But, as we have already seen, this was because of the expert's own error in calculation.29 The correct figure was such that (i.e., -1.56), even under the expert's own test it would not have been statistically significant. It follows that in both cases the standard deviation is less than -2.

It will be noted that the expert would test the results of his calculations of standard deviations both in exhibits 34a and 35a and in his buddy studies (but, significantly, not exhibits 34 and 35) by a "one-tail" test of significance and it is on the basis of this test that his opinion of statistical probability of discrimination rests. Since the findings of the plaintiffs' expert thus depend on the use of the "one-tail" test, it is necessary to understand first the difference between a "one-tail" test and a "two-tail" test and to determine under what circumstances, if any, it is proper to use a "one-tail" test. The "two-tail" test, which was the one used by the Supreme Court in Castaneda and Hazelwood, and which, as we have already noted, is the other test used in this connection, proceeds on the basis of a "null hypothesis," which was described by us in American National Bank at page 1191 as "the hypothesis that underrepresentation of a protected minority group in any sample made up of a protected and nonprotected group (binomial distribution) might be attributable to normal fluctuations of chance rather than to discriminatory design." In the application of this "one-tail" test to any compilation, however, one begins with the assumption or hypothesis, based on other evidence than that in the actual compilation being analyzed, that the defendant has been guilty of discrimination and adjust the results on the basis of that assumption. The difference in result between the two tests is significant and dramatic. Plaintiffs' expert conceded as much, and this is evident from the standards themselves for determining statistical significance under the two tests. The rule in Castaneda and Hazelwood requires standard deviations in the range of "more than two or three," and under the "two-tail" test, as often stated by statisticians "about two" standard deviations are the necessary predicate for a finding of statistical significance under the view of some social scientists. But when the "one-tail" test is used, the plaintiffs' expert testified that 1.65 standard deviations warranted a finding of statistical significance.30

Both the District Court and the plaintiffs' expert recognized that the "one-tail" test is a dramatically stricter standard for statistical significance than the "two-tail" test. In fact, the court in Brown v. Delta Air Lines, Inc., 522 F.Supp. 1218, 1229, n. 14 (S.D.Texas 1980), declared, on the basis of Dr. Hoffman's testimony to such effect in that case, that,

"Moreover, Dr. Hoffman's use of a one-tailed, rather than two-tailed, test favors the plaintiff's viewpoint even further (because with a one-sided test, it takes less of a variation from expectation to reach '.05 significance')."

A commentator has made the same observation about the "one-tail" test, describing it as " 'data mining' per se" which is "the statistician's term for manipulating data to prove a desired result." Harper, supra, 32 Hastings L.J. at 1355, n. 65, citing Freedman, Pisani and Purves, Statistics, 494-96 (1978). Still another text is more specific in its description of the "one-tail" test. In Friedman, Introduction to Statistics, 146-47 (Random House, 1972), the author explains:

"Note that, although at value of 1.96 is required to reject Ho at the 5 percent level with a two-tailed test, a value of only 1.64 is needed if a one-tailed test is used. Many investigators find it tempting to use a one-tailed probability level to facilitate obtaining 'significant' results.

"... the safest procedure in virtually all situations is to use two-tailed values. Using one-tailed values to make rejection of Ho (i.e., the null hypothesis) 'easier' serves to increase type I errors, while the size of the difference, as measured by rm is unaffected. A strict application of one-tailed values includes the danger of overlooking important results in the non-predicted direction, and any attempt to test for such outcomes leads to inaccurate probability values. Analysis of data based on two-tailed probability levels can be reported without apology, while it is almost always necessary to 'explain away' the use of one-tailed probability levels."

The District Court itself characterized the use of the "one-tailed" test in this case as a method of "beef[ing] up" the statistics and the expert himself employed the adjectives "stronger" and "more powerful" in his characterization of the test in his exposition of the test. Because of this fact (i.e., the more favorable aspect of the "beef[ed] up" test for the plaintiffs in a discrimination claim), plaintiffs' expert testified that its use in any case is conditioned upon the presence in the record of other evidence which would justify a belief or an assumption that the defendant had been guilty of discrimination. The evidence in this case which, according to the expert, justified the use of such test in this case, consisted of certain data gathered by him and incorporated in exhibits identified and introduced before introducing exhibits 34 and 35. Such evidence, at least that which it thought pertinent, was presumably that which was identified by the District Court in its findings before it accepted the results of the expert's "one-tail" test results. Whether the use of the "one-tail" test was appropriate in this case even under the expert's theory thus depends on whether this evidence on which the District Court relied was sufficient to generate a legitimate belief or assumption of discrimination. Though we do not conclude that this is a case in which the "one-tail" test should not have been used in any event, we have reviewed the evidence cited and relied on by the expert for his assumed justification for his use of the results of the "one-tailed" test.

In his assumed justification, the plaintiffs' expert relied on multiple regression studies, submitted by him in addition to exhibits 34, 35, 34a, 35a. Similar studies were offered in Stastny v. Southern Bell Tel. & Tel. Co., supra, 628 F.2d 267. The District Court commented on the reliability of such studies thus:

"Regression analysis begins with the assumption that certain independent variables in fact determine the outcome of decisions to raise pay and promote. Such assumptions are intellectually questionable and not grounded upon any solid evidence." 458 F.Supp. at 323.31

Whether this is a fair comment on regression studies is a matter we need not concern ourselves with in this case. The studies in this case are insufficient, even under the expert's faulty theory of justification, to provide a reasonable basis for an inference, hypothesis or belief that the defendant had been guilty of discrimination. We accordingly proceed to examine the studies on which presumably the District Court justified its approval of the expert's use of a "one-tail" test in establishing statistical significance by its standards in this case.

The first multiple regression of the studies to which the District Court referred in his findings was a comparison of earnings of black employees for the defendant's workforce taken as a whole. The records used by the expert, however, show that there was a greater percentage of whites in the upper pay grades than blacks. This, as the District Court found, could not have been because of any "significant difference in initial job assignments and pay grades, performance evaluations or promotion of black and white employees;" the District Court expressly found that there was no difference.32 Moreover, almost all the employees in the managerial category, which consisted of the higher pay grades, were long-time employees; in fact, the employment of many of them extended back beyond the effective date of the Act itself. In addition, the expert's studies did not show that there was any discrimination in pay rates between whites and blacks in pay grades 4 and 5. Moreover, there was a greater proportion of white employees in the higher clerical levels than of black employees, though the proportion of blacks in these pay grades was considerably above the proportion in the black force qualified for such pay grades, according to the defendant's figures. It would seem reasonable to assume that, because of the greater concentration of whites in pay grades above pay grade 5 carrying higher pay rates than pay grades 4 and 5, white employees, taken as a group throughout the defendant's work force, would receive higher wages than blacks. For this reason (the inclusion of all employees, those at the lowermost level of pay and those at the highest), regressive tables involving an overall comparison of salaries of employees have been widely dismissed by the courts as completely unreliable.

In Agarwal v. Arthur G. McKee and Co., 19 FEP Cases 503 (N.D.Cal.), aff'd. 644 F.2d 803 (9th Cir.1981), the plaintiffs, just as plaintiffs' expert here, attempted to use a regressive analysis in support of discrimination based on a comparison of salaries of minority and non-minority employees at all pay levels in an employer's work force. In finding the study meaningless, the Court said (p. 512):

"[P]laintiff's regression analyses contain a number of defects. Plaintiff failed to treat salary as a function of job position and salary grade. Furthermore, plaintiff treated all job positions as fungible, involving equal levels of knowledge, skill, and responsibility. Therefore, plaintiff's statistics do not refute defendant's contention that salary differences between minorities and non-minorities within each job position are not substantial." (Italics added)

Valentino v. U.S. Postal Service, 511 F.Supp. 917, 957 (D.C.D.), aff'd, 674 F.2d 56 (D.C.1982), involved the use of a like comparison of salary difference for the defendant's workforce as a basis for a claim of racial discrimination in salary. Again referring to the erroneous treatment of "all jobs as fungible" and finding the study unreliable, the Court said:

"Plaintiff treated the wide variety of positions in USPS Headquarters at level 17 and above as fungible items. For example, her regression compared the position of the Postmaster General to that of his secretary. Common sense dictates that these positions are not comparable, are not fungible, and that any difference in salary cannot be presumed to be the result of sex discrimination by the USPA."33

Again, the Court in Vuyanich v. Republic Nat. Bank of Dallas, 505 F.Supp. 224 at 280 (N.D.Tex.1980), dismissed as valueless salary comparisons between white and black employees at all levels of employment on the issue of discrimination. The Court said:

"Not surprisingly, where an employer has employees in differing occupations and of different backgrounds, a simple comparison of the average wage of all white employees and all black employees (or all male employees and all female employees) will not be enough to prove salary discrimination. See, e.g., Pouncy v. Prudential Insurance Co. of America, supra at 449 [499 F.Supp. 427] 23 Emp.Prac.Dec. n. 66, at 16,751 ('The Court believes that the proper inquiry in an analysis of salaries by race should focus on whether black and white employees with the same tenure at the same job level are paid the same salaries') ...."

Keely v. Westinghouse Electric Corp., 404 F.Supp. 573 (E.D.Mo.1975) is another case where a salary comparison was attempted to be used to support an inference of discrimination. The Court dismissed, as "meaningless" the evidence, saying:

"Plaintiff has submitted evidence showing that blacks were earning less on the average than were defendant's white employees. Such statistics, however, are meaningless without more. Were evidence produced which showed that black employees with the same length of employment and the same qualifications were paid lower rates for similar jobs, or that no black employee earned more than any white employee, this Court might feel compelled to conclude that the disparity was due to racial discrimination."34 [p. 578]

Similarly, in this case, the salary comparison referred to in the District Court's findings and used by the plaintiffs' expert as a basis for his "hypothesis" or belief of discrimination against the defendant treated the vice president of the bank as comparable to the cafeteria waitress and considered the eight officers (all of whom are above the pay grade 16) as comparable with employees in pay grade 3, the lowest pay grade in the existing workforce of the defendant. Many other comparisons, almost as lop-sided, could be cited as demonstrating the same flaw. Just as the court ruled in Valentino and in Ste. Marie, this table of salary comparison cannot be used to give support to any presumption of discrimination. Moreover, we are concerned solely with alleged discrimination in pay grades 4 and 5. It would have been appropriate, however, if we had compared wages in those pay grades alone. See Vuyanich, supra, 505 F.Supp. 224. Yet the fact is that the District Court found that there was no discrimination in pay in any specific pay grade, including pay grades 4 and 5.

A second set of statistics cited by the District Court relates to the assignment of employees "to cleaning positions and the Cafeteria" and to assignments in pay grades 6 to 14. We are unable to find any basis for presuming discrimination on this basis since the District Court both in its "Memorandum of Decision" and in its findings concluded that there was no pervasive evidence of discrimination in the job assignment of employees. Moreover, the employment records reviewed by the plaintiffs' expert demonstrated that over 85% of the bank employees employed in service and cafeteria jobs had requested that type of work and/or that their work experience before hiring was in that type of work. So far as those blacks employed in this work who had not requested jobs in those departments, they, with hardly an exception, had submitted records that indicated work experience only in those areas and, in particular, had no background in clerical work. It is agreed in the District Court's findings that initial employees when employed, were assigned, whether they were black or white, on the basis of "educational background and prior work experience." In view of the District Court's finding of no discrimination in assignments, it is difficult to understand how the expert could have relied on assignments for his presumption of discrimination.

The District Court also made a finding, based on some data prepared by the expert, that "Fifty-three (53) percent of the black employees, however, were in pay grades 6 and below, as compared with 26 percent of the white employees." This, too, was cited and relied on as authority for a belief that the defendant had engaged in discrimination. What, however, this finding fails to note is that, for pay grades above grade 6 added qualifications are required. There is in the record the percentage of black employees in pay grades 7 to 13 which are the pay grades requiring special qualifications. The percentage of black employees of the defendant in such pay grades is 17.5 percent as compared with a 10.4 percentage of blacks having such special qualifications in the qualified black labor force in the Charlotte employment area as shown by official Labor Department figures. Moreover, the District Court itself found as a fact, and the plaintiffs have not disputed such finding, that there has been no discrimination in promotions or hirings in pay grades 6 and above or in job assignments or in pay in such grades. If this fact be assumed (and the District Court so found) it would appear impossible to derive an inference of discrimination based on a comparison of black and white employees in those pay grades. The situation here is similar to that in Pouncy v. Prudential Ins. Co. of America, 668 F.2d 795, 801-02 (5th Cir.1982), in which the plaintiffs in a discrimination case made a like contention based on a statistical table which "show[ed] that, on the whole, blacks [were] overrepresented in the lower levels of Prudential's work force." The Court dismissed such evidence as meaningless, saying:

"But this might result from any number of causes. Absent proof that the disparate impact is caused by one or more of the challenged employment practices, we do not require the employer to justify the legitimacy of any (or all) employment practices."

In this case, any disparity reflected in this table could not, as we have said, be the result of any discrimination in hiring or assignment; the District Court has found the absence of any such discrimination and the plaintiffs have not excepted to that finding. If it be said it is due to discrimination in promotions out of pay grades 4 and 5, the statistical evidence in the tables submitted by the plaintiffs will not support such a finding, if we follow the Hazelwood and American National Bank test (statistical deviations in the range of "about two or three," with only the existence of at least three deviations being sufficient for an absolute inference of discrimination).

The District Court noted that there was a disparity in the average time rate of promotions out of pay grade 4 (but not pay grade 5 ) between white and black employees in the expert's studies. However, the studies compiled by plaintiffs' expert showed the comparative rates in promotion overall of black and white employees and in pay grades 4 and 5 for the relevant years. Specifically, the expert testified that black employees had less or equivalent time at the bank in each of the grade levels above grade 4 (that is, for pay grades 5 and above), in all the years in issue and that "the promotion rate of blacks [all grades included] was greater than for whites" in the same years.

Moreover, the studies prepared by the expert himself in connection with his regressive studies, indicated quite clearly why there should have been a disparity in promotion of blacks out of grade 4. These studies showed that a far greater proportion of black hirees brought to their employment an experience in service and cafeteria jobs than white hirees and, conversely, the white hirees brought a far greater work experience in clerical work than black employees. Since it was found that job assignments for hirees were based on educational background and work experience, it was reasonable to expect that a far greater proportion of black hirees than white hirees would be assigned to service and cafeteria jobs and that a far greater proportion of white employees than black employees would be assigned to clerical jobs. And these expected assignments of black and white hirees were justified by the admitted fact that a far greater proportion of black hirees than white hirees requested service and cafeteria jobs and a far greater proportion of white hirees than black hirees applied for clerical jobs.

In a bank where the opportunities for promotion primarily were in the clerical or office fields, it is to be expected that those whose work and experience were in those fields would have an advantage in promotions over employees in service and cafeteria jobs.35 Experience in a cafeteria or in cleaning hardly offers training that qualifies one for clerical work in a bank such as a Federal Reserve Bank. These are the very reasons which prompted the Court in Ste. Marie to find flawed statistics showing disparity in the employment of whites over minorities in a discrimination case. In that case, Judge Friendly said:"Plaintiff's statistics were hopelessly flawed by the lumping of these secretarial jobs into the clerical category, since these positions did not offer the incumbents the opportunity to acquire the skills and experience that would enable them to qualify for promotion to technical and still less to managerial posts. There is no principle requiring an employer following a policy of promoting from within to make this applicable across the board rather than only to those employee groups whose work gives them the opportunity to acquire the skills needed for promotion. Yet plaintiff's statistics gave the same weight to failure to promote secretaries and typists to posts requiring specialized substantive knowledge and experience as they did to failure to promote women working in other clerical positions that would permit them to obtain the essential skills. This methodology failed to heed the warning in Hazelwood School District v. United States, supra, 433 U.S. at 308 n. 13, 97 S.Ct. at 2742 n. 13: 'When special qualifications are required to fill particular jobs, comparisons to the general population (rather than to the smaller group of individuals who possess the necessary qualifications) may have little probative value.' " [pp. 400-01]

See also Pouncy v. Prudential Ins. Co., supra, 668 F.2d at 804. The difficulty in our case, as it was in Ste. Marie, is that the difference in the experience acquired on the job assigned in a non-discriminatory way may reasonably account for promotion to higher clerical jobs, rather than discrimination. These considerations are the manifest reasons for the disparity in promotions between whites and blacks at the grade 4 level in this case. They should have been noticed and taken account of by the plaintiffs' expert in his calculations as well as by the District Court. They do not justify a presumption of discrimination in connection with promotions at pay grade 4.

It follows that, assuming for the moment, it was permissible for plaintiffs' expert to look to other statistics in order to justify the use of the "one-tail" test which was favorable to the plaintiffs, the fact is that none of the statistics he relied on fairly would have justified the assumption of discrimination on which he predicated his right to use this "one-tail" test. We repeat, however, that we are not persuaded that it is at all proper to use a test such as the "one-tail" test which all opinion finds to be skewed in favor of plaintiffs in discrimination cases, especially when the use of all other neutral analyses refutes any inference of discrimination, as in this case.

Finally, in summary, the expert's opinion on statistical significance based as it is on the use of both the hypergeometric test and the "one-tail" standard of statistical significance, which, as we have seen, is the linchpin for the District Court's conclusion of discrimination, depends for any meaning on the complete acceptance of a sequence of dependent adjustments made in the relevant data and applicable formulae, as demonstrated by our review of the plaintiff's statistical evidence, coupled with a complete disregard of all contrary conclusions evident in the statistical evidence. To recapitulate, we begin this sequential review with exhibits 34 and 35, as prepared initially by the plaintiffs' expert (never discussed in the District Court's Findings) which use the actual promotions of blacks over the relevant period 1974-78 in the pertinent pay grades of 4 and 5. If we apply in these exhibits the binomial test, followed in Castaneda and Hazelwood and accepted by us in Moultrie, and American National Bank, supra, 652 F.2d at 1193, n. 12, for determining the applicable standard deviations for measuring any disparity in promotions out of pay grades 4 and 5, the result in standard deviations for disparity in promotions of black employees stated in both pay grades show a variance well below 2 (i.e., -1.52 in pay grade 4 and -1.24 in pay grade 5).36 Even if we use the hypergeometric, rather than the binomial, test, as did plaintiffs' expert in his statement in his tables 34 and 35, the result in standard deviations is below 2 (i.e., 1.90 in pay grade 4 and 1.56 in pay grade 5). There was thus no basis for a finding of statistical significance as a result of tables 34 and 35, irrespective of whether one employs the binomial or the hypergeometric test. These tables are not mentioned in the Findings proposed by the plaintiffs and adopted by the District Court.

The expert's presentation then shifts to exhibits 34a and 35a. In these exhibits, accepted by the District Court and included in its Findings, the expert, has, however, changed the number of employees in the sample, both overall and divided between white and black, and more importantly, changed the number of actual black promotions made during the relevant years. In these samples, a number of employees in the pay grades and a number of the black promotions are eliminated. With the pertinent data thus abbreviated, the expert came up with a new set of results in standard deviations. He did not state these results in terms of a binomial test, under which the standard deviations in pay grade 4 would have been barely over 2 (i.e., -2.07) and less than 2 in pay grade 5 (i.e., -1.45).37 On the contrary, he used the hypergeometric test in his tables 34a and 35a. The use of the hypergeometric test in this context is not warranted under the conditions stated by some respected commentators. Under this hypergeometric test, however, the standard deviations for pay grade 4 were -2.69 and -1.87 for pay grade 5. Even if this test result is accepted, there would be no basis whatsoever for inferring discrimination in pay grade 5, and, under our rule as stated in American National Bank, ("... courts of law should be extremely cautious in drawing any conclusions from standard deviations in the range of one to three," 652 F.2d at 1192), a finding of statistical significance, much less legal significance, in pay grade 4 would be accepted only with extreme caution, an injunction particularly apt in view of the undisputed fact that a disproportionate number of black employees in this pay grade were employed in the cafeteria and service departments from which promotion into clerical jobs at the higher pay levels would be less likely than for those in that pay grade having clerical experience. See Ste. Marie, supra, 650 F.2d at 401.

The expert apparently recognized and even conceded that, for the reasons already given, statistical significance could not be derived from the results of either the binomial or the hypergeometric test under exhibits 34, 35, 34a, 35a or the "buddy" test study which he conducted (in which the standard deviations were less than 2) but he testified that he was entitled to use a 1.65 standard of standard deviations as a test of statistical significance (i.e., the "one-tail" test). The propriety of the use of such a test, admittedly favorable to the plaintiffs, depended, even according to plaintiffs' expert, on the right of the expert to assume on the basis of other data accumulated by him that the defendant had been guilty of discrimination and to analyze the results of exhibits 34, 35, 34a and 35a on that basis. As we have said, the facts and data cited by the expert and included in the District Court's Findings simply did not justify such a belief or assumption, and therefore, did not justify the use in this case of a "beef[ed] up" "one tail" test, to use the District Court's own description of the test. But it is only by the use of such a test that he reached a result of standard deviations of -2.69 in pay grade 4 and of -2.01 in pay grade 5, (both of which are below the "safe" figure of -.3). In short, plaintiffs' expert has achieved his opinion of statistical significance by discarding all tests except one which he has admitted in Brown (522 F.Supp. 1218) favors the plaintiffs, and even where he limited himself to that test, he has reached a result which we said in American National Bank was to be accepted with extreme caution. We are of the opinion that the District Court was in clear error in accepting the opinion of plaintiffs' expert on statistical significance when that opinion rested on such skewed analyses and which disregarded the far more reliable tables, (i.e., tables 34 and 35), which demonstrated no basis for a finding of statistical significance, much less legal significance.

Before concluding the class action aspect of this case, we should observe that the defendant offered considerable expert testimony in support of its defense of no discrimination, little of which was noticed or discussed by the District Court in its Findings, and then only in an effort to support its finding of discrimination. The evidence submitted by the defendant was clearly relevant to the defendant's defense that it had never subjected its black employees, either in pay grades 4 and 5, or in any other pay grades, to discriminatory treatment and deserved consideration by the District Court. It consisted largely of various statistical studies made by defendant's expert. One of these studies showed, for instance, that between 1966 and 1978, white employment at the bank dropped from 82.3% to 64.3% while black employment increased from 17.7% to 35.7% and that in the period between 1974 and 1978 thirty-five percent of all employees hired by the bank were black, although the black representation in the relevant labor market was but twenty percent.

In another chart, the bank's employees were divided into groups based on type of job corresponding to the Labor Department classifications. The percentage of blacks in each group classification in the Charlotte metropolitan area as shown by the relevant Labor Department statistics was ascertained. Pay grades 3 through 6 under the defendant's employment procedures were classified as a group which "require[d] relatively little training and persons from the general labor market [could] perform at those jobs" in that group. 46.4% of the black employees in 1978 were within the class. This was practically two times the available black employees in the general labor force in the Charlotte area (i.e., 19.9%). The second grouping covered the employees in pay grades 7 through 13 qualifying within the classification of "labor force clerical employees." In this group were 16.2% of the bank's black employees as contrasted with available blacks meeting the qualifications of such group in the relevant labor market of 10.4%. Again, the percentage of black employees in these pay grades is considerably more than the available qualified blacks in such group in the Charlotte area. The final group embraced pay grades 14 through 16 plus all officers and it was classified in the labor statistics as "non-farm managers and administrators." In this group there was a single black in defendant's labor force but because of the small number involved, this represented 5.6% of those employed in such group as contrasted with 3.75% among the available qualified blacks in such classification in the relevant labor area. These comparisons, as set forth in these tables, follow the model set forth in Hazelwood School District v. United States, 433 U.S. 299, 97 S.Ct. 2736, 53 L.Ed.2d 768, under which the Supreme Court measured disparity by comparing the percentage of black hirees to the percentage of blacks available in the relevant labor market. This method of analysis has been declared to be probative of both an employer's actual hiring practices and its recruiting practices. Clark v. Chrysler Corp., 673 F.2d 921, 929 (7th Cir.1982). In this case, the fact that blacks were more heavily represented at all levels than their representation in the relevant qualified labor pool would be probative of an absence of discrimination in recruiting, in assignments and in promotions.

Another chart of the defendant traced the promotion rate of black and white employees employed at all pay grades in the period 1974 to 1978, within the fifteen-month period after they had been hired. This data showed the percentage of black employees in the entire workforce of the bank in the 1974-78 period, was 33%; during this period black employees received 35% of all the promotions. The defendant's expert calculated from his charts that "[t]he probability of [a] white [employee] being promoted [in the 1974-78 period] was about 18%, whereas the probability of a black being promoted [was] 20%." This calculation was not questioned by the plaintiffs. Moreover, all this information on rate of promotion was verified in the charts developed by plaintiffs' expert. Thus, looking at the years 1974-78, the plaintiffs' statistics established that, except in grade 4, blacks had either less or equivalent time than whites in every pay grade from 5 on. Thus, in pay grade 5 in 1974, whites remained in that pay grade 45 m